CDNcheck: How ProjectDiscovery Built an Offline CDN Detector That Runs Without API Calls
Hook
Most CDN detection tools ping external APIs for every lookup. cdncheck flips this model entirely: it compiles provider data into a local database and performs all detection in-memory, making it possible to process thousands of IPs per second without rate limits or internet dependencies.
Context
Security researchers and penetration testers constantly need to identify whether a target is behind a CDN, cloud provider, or web application firewall. This information is critical for attack surface mapping—knowing that example.com sits behind Cloudflare changes your reconnaissance strategy entirely. Traditional approaches involve DNS lookups, WHOIS queries, or commercial API services like ipinfo.io. Each method has tradeoffs: DNS queries are slow for bulk operations, WHOIS data requires expert interpretation, and commercial APIs impose rate limits and costs.
ProjectDiscovery, the team behind popular security tools like Nuclei and httpx, built cdncheck to solve this workflow friction. Instead of calling external services, cdncheck pre-compiles a database of IP ranges, ASN numbers, and CIDR blocks from public provider sources. The tool downloads Cloudfront’s IP list, Fastly’s public ranges, and dozens of other provider endpoints, then indexes them for instant local lookups. The result is a utility that trades guaranteed real-time accuracy for speed, zero dependencies, and the ability to run completely offline.
Technical Insight
cdncheck’s architecture revolves around a two-phase design: compilation and detection. The compilation phase, handled by the generate-index program, transforms a YAML configuration file into an optimized JSON database. Let’s examine how providers are defined in provider.yaml:
cdn:
asn:
leaseweb:
- AS60626
urls:
cloudfront:
- https://d7uri8nf7uskq.cloudfront.net/tools/list-cloudfront-ips
fastly:
- https://api.fastly.com/public-ip-list
cidr:
akamai:
- "23.235.32.0/20"
- "43.249.72.0/22"This declarative format supports three detection methods: URL-scraped ranges (fetched from provider endpoints like Cloudfront’s public IP list), static CIDR blocks (manually curated ranges), and ASN-based detection (matching autonomous system numbers). The generate-index tool fetches remote URLs, parses responses, and consolidates everything into sources_data.json, which ships with the compiled binary.
The detection phase uses Go’s net package for IP range matching. When you query an IP address, cdncheck performs CIDR containment checks against its compiled database. Here’s how you’d use it as a library:
package main
import (
"fmt"
"net"
"github.com/projectdiscovery/cdncheck"
)
func main() {
client := cdncheck.New()
ip := net.ParseIP("173.245.48.12")
// Check if IP belongs to a CDN
matched, val, err := client.CheckCDN(ip)
if err != nil {
panic(err)
}
if matched {
fmt.Printf("%v is a %v\n", ip, val)
} else {
fmt.Printf("%v is not a CDN\n", ip)
}
}Beyond IP matching, cdncheck implements CNAME-based detection for providers that use distinctive DNS patterns. This happens in other.go through simple map lookups:
var cdnCnameDomains = map[string]string{
"cloudfront.net": "amazon",
"amazonaws.com": "amazon",
}When you pass a domain to cdncheck, it appears to resolve CNAME records and check if any hop matches these hardcoded patterns. This catches cases where IP-based detection fails—like dynamically allocated cloud instances that don’t fall within known CIDR ranges.
The CLI tool wraps this library with filtering capabilities. You can match specific providers (-mcdn cloudflare) or exclude them (-fwaf akamai). The -resp flag includes provider names in output, while -jsonl formats results as JSON lines for pipeline processing.
This design philosophy—compile once, query locally—makes cdncheck extremely fast for batch operations. During penetration tests, you might scan thousands of subdomains and need instant CDN classification without hammering external APIs or worrying about rate limits. The tradeoff is staleness: your database is only as current as your last generate-index run or software update.
Gotcha
cdncheck’s biggest limitation is its reliance on manually maintained provider lists. If Cloudflare allocates a new IP range tomorrow, your local database won’t know about it until someone updates provider.yaml, regenerates the index, and you pull the latest release. There’s no automatic discovery mechanism. For security assessments where 100% accuracy is critical, this staleness can produce false negatives—you might miss that a target moved to a CDN last week.
The CNAME detection method appears to be less flexible than the YAML-based approach. Adding a new CDN that uses CNAME patterns requires editing Go source code in other.go and recompiling, rather than just updating a configuration file. This creates friction for community contributions—adding a new URL-scraped provider is a simple YAML edit, but CNAME-based providers demand code changes. Additionally, niche or regional CDN providers won’t appear in the database unless someone explicitly adds them, so coverage is biased toward major global providers like Cloudflare, AWS, and Fastly.
Verdict
Use cdncheck if you’re performing security reconnaissance, penetration testing, or attack surface mapping where speed and offline operation matter more than real-time accuracy. It’s especially valuable in ProjectDiscovery toolchains—piping httpx results through cdncheck to filter targets, or using it as a library within custom Go scanners. Choose it for batch processing scenarios where you need to classify thousands of IPs without API costs or rate limits. Skip it if you need guaranteed up-to-date provider data for compliance reporting, require detection of newly launched or regional CDN providers not in the database, or prefer vendor-supported commercial solutions with SLAs. Also skip it if your workflow already uses commercial IP intelligence APIs that provide richer metadata beyond CDN detection—cdncheck is laser-focused on one task and doesn’t attempt to replace comprehensive WHOIS or ASN databases.
