l9explore: Building Security Scanners with Unix Pipelines

Hook

Most security scanners are monolithic beasts that try to do everything. l9explore takes the opposite approach: it does one thing well and expects you to pipe data through it like it's 1970.

Context

Security scanning has largely abandoned Unix philosophy in favor of all-in-one solutions. Tools like Nessus, OpenVAS, and even Metasploit operate as monolithic platforms where data flows through internal databases and APIs. This makes them powerful but inflexible—you can't easily chain scanning tools together or insert custom processing steps without wrestling with their plugin APIs.

LeakIX built l9explore as part of their l9 toolchain, which takes security scanning back to basics: small, composable tools connected through pipes. The problem they're solving isn't just "scan for vulnerabilities"—it's "make security scanning programmable." Instead of clicking through a web UI or learning a domain-specific language, you compose scanning pipelines using standard shell tools. l9explore specifically handles the "explore" phase: taking identified network services (from l9tcpid) and probing them for misconfigurations, exposed data, and vulnerabilities. It's designed for the operator who wants to script their reconnaissance and integrate scanning into automated workflows.

Technical Insight

The architecture of l9explore centers on a standardized JSON format called l9format that flows between tools via stdin and stdout. This isn't just lazy Unix piping—it's a deliberate contract that enables tool composition. When l9tcpid identifies that port 27017 is running MongoDB, it emits l9format JSON. l9explore reads this, loads the appropriate MongoDB plugins, executes checks, and outputs enriched l9format JSON with findings.

The plugin system operates in three conceptual stages: open, explore, and exfiltrate. Open plugins perform basic connectivity checks—can we authenticate? Is the service really what we think it is? Explore plugins dig deeper, enumerating databases, checking configurations, testing for known CVEs. Exfiltrate plugins (marked as WIP) are designed to actually pull data, though this stage remains incomplete. Each plugin registers itself with stage metadata and protocol filters.

Here's what a simple pipeline looks like in practice:

# Scan a CIDR, identify protocols, then explore them
echo "192.168.1.0/24" | ip4scout | l9tcpid | l9explore | l9filter

That single pipeline does port scanning, protocol identification, vulnerability checking, and filtering—all through composable tools. Each tool is independent and can be swapped out.

Internally, l9explore uses Go's goroutines for concurrent plugin execution. When a service matches multiple plugins, they run in parallel with configurable thread limits to prevent resource exhaustion. The plugin interface is surprisingly simple:

type Plugin interface {
    Initialize() error
    GetStage() string  // "open", "explore", or "exfiltrate"
    GetProtocol() string  // "http", "mongodb", "redis", etc.
    Run(service *l9format.Service) (*l9format.Event, error)
}

Plugins receive an l9format.Service struct containing IP, port, protocol metadata, and banner information. They return an l9format.Event containing any findings—credentials, exposed endpoints, CVEs, whatever they discover. The simplicity here is intentional: plugins don't need to handle network orchestration, output formatting, or pipeline coordination.

The HTTP plugin demonstrates the architecture's flexibility. It doesn't just check for open webservers—it probes for specific exposures like Laravel Telescope debug panels, exposed .env files, phpinfo pages, and directory listings. Each check is a separate function, but they all run through the same plugin harness:

// Simplified example of HTTP plugin structure
func (p *HttpPlugin) Run(service *l9format.Service) (*l9format.Event, error) {
    client := &http.Client{Timeout: 10 * time.Second}
    
    // Check common exposure paths
    paths := []string{
        "/.env",
        "/telescope",
        "/phpinfo.php",
        "/.git/config"
    }
    
    for _, path := range paths {
        resp, err := client.Get(fmt.Sprintf("http://%s:%d%s", 
            service.IP, service.Port, path))
        if err != nil {
            continue
        }
        
        if resp.StatusCode == 200 {
            return p.createEvent(service, path, resp)
        }
    }
    
    return nil, nil
}

The database plugins take this further. The MongoDB plugin doesn't just check for authentication—it enumerates databases, checks for specific collections that commonly contain sensitive data (users, customers, payments), and can calculate statistics about exposed records. The Elasticsearch plugin similarly checks cluster health endpoints and enumerates indices.

What makes this architecture powerful is that findings are cumulative. As JSON flows through the pipeline, each tool adds its discoveries without removing previous data. By the time l9explore's output reaches l9filter, you have a complete picture: which IP, which port, which protocol, which vulnerability, all in structured JSON ready for additional processing or storage.

The threading model deserves attention too. l9explore doesn't spawn unlimited goroutines—it uses a worker pool pattern with configurable concurrency. This matters when you're scanning thousands of hosts; unbounded concurrency would exhaust file descriptors and memory. The tool balances throughput with resource constraints, making it suitable for running on modest VPS instances during reconnaissance.

Gotcha

The multi-stage plugin system is marked as work-in-progress, and it shows. The exfiltrate stage—which would actually dump discovered data—isn't fully implemented. This means l9explore will tell you about an exposed MongoDB database but won't automatically extract the data. You'll need to write custom scripts or use other tools for that phase.

More critically, l9explore requires careful operational discipline around authorization. The repository's disclaimer isn't legal boilerplate—it's a genuine warning. The tool is designed to find exposed data, and using it against systems you don't own or have authorization to test is illegal in most jurisdictions. There's no built-in safeguard preventing you from scanning arbitrary internet ranges. The responsibility sits entirely on the operator.

Protocol coverage is limited to whatever plugins exist. While common protocols (HTTP, MongoDB, Redis, Elasticsearch, MySQL) are covered, less common services require writing your own plugins. The plugin API is documented but not extensively—you'll need to read existing plugin source code to understand patterns and conventions. There's also no plugin marketplace or central repository; discovering community plugins means searching GitHub.

Verdict

Use if: You're already thinking in Unix pipelines and want security scanning that fits that model. You need lightweight, scriptable reconnaissance that integrates with existing automation. You're comfortable reading Go code to understand plugin behavior, or you need to scan specific protocols where l9explore has plugins and you value speed over comprehensive scanning. You have proper authorization for everything you scan. Skip if: You want an all-in-one scanner with GUI and reporting dashboards. You need mature data exfiltration capabilities beyond detection. You're scanning protocols without existing plugins and don't want to write Go code. You need extensive documentation and community support—l9explore is powerful but assumes operator expertise and comfort with command-line workflows.