XRay: Building a Reconnaissance Pipeline That Bridges Passive OSINT and Active Enumeration

Hook

Most reconnaissance tools force you to choose between stealthy passive enumeration or thorough active scanning. XRay attempted to build a pipeline that does both sequentially, creating a workflow that trades OPSEC for automation.

Context

Before comprehensive reconnaissance frameworks emerged, security researchers cobbled together scripts that queried DNS records, called Shodan APIs, extracted certificate data, and grabbed service banners—all as separate manual steps. Each tool lived in isolation: dnsrecon for subdomains, shodan-cli for historical data, nmap for service detection. The cognitive overhead was significant, and aggregating results meant writing custom parsers for each tool's output format.

XRay emerged in this fragmented landscape as an opinionated solution to the reconnaissance workflow problem. Built by evilsocket (creator of bettercap and other offensive security tools), it treated recon as a data pipeline: start with a domain, discover all subdomains through DNS bruteforcing, enrich each finding with Shodan's historical port/service data and ViewDNS records, then actively probe discovered endpoints with protocol-specific banner grabbers. The goal was eliminating the context-switching tax that came from running a dozen different tools and manually correlating their outputs. Everything would feed into a unified session file that persisted across runs and rendered through a built-in web UI.

Technical Insight

XRay's architecture follows a classic producer-consumer pattern with three distinct stages. The first stage is subdomain enumeration, where the tool reads from a wordlist (like the included lists from SecLists) and bruteforces DNS records. Here's how the core enumeration loop structures concurrent workers:

// Simplified from the actual codebase
func (s *Session) bruteforceSubdomains(domain string, wordlist []string) {
    jobs := make(chan string, len(wordlist))
    results := make(chan *Subdomain, len(wordlist))
    
    // Spawn consumer workers
    for w := 0; w < s.Consumers; w++ {
        go func() {
            for candidate := range jobs {
                target := candidate + "." + domain
                if ips, err := net.LookupHost(target); err == nil {
                    results <- &Subdomain{
                        Host: target,
                        IPs: ips,
                    }
                }
            }
        }()
    }
    
    // Feed the workers
    for _, word := range wordlist {
        jobs <- word
    }
    close(jobs)
}

This concurrent model defaults to 16 workers, each pulling subdomain candidates from a shared channel and performing DNS lookups. It's straightforward goroutine-based concurrency without fancy work-stealing or backpressure mechanisms—simple but effective for I/O-bound DNS queries.

The second stage is where XRay differentiates itself: API enrichment. For each discovered subdomain, it queries Shodan's API to retrieve historical port/service data and ViewDNS for IP history. This passive intelligence layer means you're not directly port-scanning the target yet, reducing immediate detection risk. The Shodan integration is particularly clever because it reveals services that might be temporarily offline or firewalled during your scan window but were exposed historically.

The third stage deploys protocol-specific banner grabbers. XRay includes custom implementations for HTTP/HTTPS (with TLS certificate parsing), SSH, FTP, SMTP, MySQL, and other common services. The HTTP grabber is especially sophisticated—it doesn't just fetch the root page but extracts title tags, server headers, and parses TLS certificates for Subject Alternative Names (SANs). This certificate mining creates a recursive discovery mechanism: if a certificate lists additional subdomains not in your wordlist, those get added back into the enumeration queue. This certificate-chain-walking technique often uncovers internal infrastructure that wouldn't appear in external DNS wordlists.

The session architecture deserves attention. XRay serializes all findings into a JSON session file that persists between runs. This means you can interrupt a long-running scan, modify parameters, and resume without losing previous work. The session becomes the source of truth, feeding the web UI that renders an interactive graph of your target's attack surface. The UI runs as a local HTTP server, serving a client-side JavaScript application that visualizes subdomain relationships and service distributions.

One underappreciated design choice is the domain-specific grabber abstraction. Rather than relying on generic socket connections, XRay implements protocol handshakes correctly—SSH version exchange, SMTP HELO sequences, MySQL greeting packets. This produces higher-fidelity fingerprints than raw banner grabs but comes with complexity: each protocol needs custom parsing logic, and protocol changes can break grabbers.

Gotcha

The elephant in the room is that XRay is officially deprecated. The author explicitly recommends using 'legba,' a Rust reimplementation with expanded capabilities. This isn't just legacy code you could fork and maintain—it represents architectural decisions that made sense in 2017 but don't align with modern reconnaissance requirements. The codebase hasn't seen meaningful updates in years, which means no support for newer TLS versions, no HTTP/2 or HTTP/3 handling, and no adaptations for cloud-native infrastructure patterns like CDN-fronted applications or containerized deployments.

The OPSEC implications are severe for any engagement where stealth matters. XRay makes unproxied DNS queries directly from your IP address, creating an easily attributable signature of enumeration activity. The active banner grabbing phase connects to every discovered service sequentially, generating logs on target systems with consistent source IPs and timing patterns that trivial IDS rules would flag. There's no Tor integration, no DNS-over-HTTPS option, no traffic shaping or jitter in connection timing. For authorized penetration tests where you've already signed engagement letters, this is fine. For bug bounty programs with unclear rules of engagement or nation-state-level adversary scenarios, it's disqualifying.

The Shodan API dependency creates both cost and capability constraints. Without an API key, XRay degrades to basic DNS enumeration and active probing—essentially a worse version of existing tools. With an API key, you're subject to Shodan's rate limits and pricing tiers. For organizations scanning large IP spaces regularly, this becomes a recurring operational expense tied to a third-party service whose API terms could change.

Verdict

Use if you're maintaining existing reconnaissance workflows that already depend on XRay's session format and web UI, you're conducting authorized internal security assessments where stealth isn't a requirement, or you're studying reconnaissance pipeline architecture for educational purposes—the codebase is readable Go that demonstrates producer-consumer patterns clearly. Skip if you're starting a new reconnaissance capability (use the author's recommended 'legba' successor instead), you need OPSEC-conscious tooling for sensitive engagements (the unproxied DNS queries and predictable connection patterns make detection trivial), you require active maintenance and modern protocol support (HTTP/2, contemporary TLS, cloud service fingerprinting), or you prefer specialized tools over all-in-one solutions (ProjectDiscovery's subfinder + httpx + nuclei pipeline is more modular and actively developed). XRay solved a real workflow problem in its era but has been architecturally superseded.