EyeWitness: How Virtual Environments Solved Python’s PEP 668 Problem for Red Teams
Hook
Python security tools broke across Kali Linux and Ubuntu 22.04+ when PEP 668 banned system-wide pip installs. EyeWitness solved it by rebuilding around virtual environments—without Docker.
Context
Web reconnaissance during penetration tests follows a predictable pattern: enumerate hosts with Nmap, identify web services, manually browse hundreds of URLs to find vulnerable applications. The manual browsing step is soul-crushing—clicking through login pages, 404s, and default Apache installations while searching for actual attack surface. EyeWitness emerged to automate this drudgery by taking screenshots, capturing server headers, and flagging default credentials across entire IP ranges.
But Python-based security tools hit a wall when Debian and Ubuntu adopted PEP 668 in 2023, preventing pip from installing packages system-wide. Kali Linux inherited this restriction. Tools that worked for years suddenly threw “externally-managed-environment” errors. EyeWitness responded with a complete architectural shift to virtual environments, turning a compatibility crisis into a cross-platform advantage. The tool simultaneously migrated from deprecated PhantomJS to Selenium with Chromium, modernizing its browser automation stack while solving the installation chaos.
Technical Insight
EyeWitness uses headless Chromium via Selenium WebDriver to render JavaScript-heavy web applications that simpler tools miss. The architecture centers on three components: a URL validation pre-flight system, adaptive threading based on CPU cores and available memory, and SQLite-backed resume capability for interrupted scans.
The virtual environment isolation strategy sidesteps PEP 668 entirely by creating a self-contained Python installation in eyewitness-venv/. Setup scripts detect the operating system, install Chrome/Chromium through native package managers, then bootstrap a virtual environment with all dependencies. Here’s how the Linux setup works after running setup.sh:
# Activate the isolated Python environment
source eyewitness-venv/bin/activate
# Scan a list of URLs with custom threading
python Python/EyeWitness.py -f targets.txt --threads 15 -d ./output
# Process Nmap XML output directly
python Python/EyeWitness.py -x nmap_scan.xml
# Single URL scan
python Python/EyeWitness.py --single https://example.com
# Use proxy for scanning (separate flags)
python Python/EyeWitness.py -f targets.txt --proxy-ip 127.0.0.1 --proxy-port 8080The adaptive resource management automatically calculates thread count as 2x CPU cores (capped at 20 threads) and monitors memory/disk space during execution. This prevents the classic problem of spawning 100 Chromium instances and watching your system grind to a halt. The tool validates URLs before launching browsers—checking for proper formatting, resolving DNS, and filtering duplicates—which eliminates wasted browser processes on malformed input.
Screenshot capture uses Selenium’s headless mode to capture the initial page load after JavaScript execution. Server header extraction provides information about the web server being scanned. The default credential detection identifies known default credentials if possible based on the application fingerprint—it appears to check for known vulnerable applications and flags them in the HTML report.
SQLite persistence stores scan progress and results in a local database. If you interrupt a 5,000-URL scan at item 3,247, EyeWitness resumes from that exact position when restarted with --resume /path/to/output/ew.db. This enables continuation of long-running scans without starting over.
The cross-platform consistency comes from using identical virtual environment patterns on Windows, Linux, and macOS. Windows uses eyewitness-venv\Scripts\activate.bat while Linux/macOS use source eyewitness-venv/bin/activate, but the underlying Python environment structure remains identical. This eliminates “works on my machine” scenarios when sharing reconnaissance playbooks across teams.
Configuration file support allows saving preferred settings like thread count, timeout values, and user-agent strings to avoid typing the same flags repeatedly. The progress tracking displays a real-time ETA based on average request completion time, helping you decide whether to grab coffee or go to lunch while scanning enterprise networks.
Gotcha
Docker support is listed as “currently in development” and non-functional, which creates friction for teams standardizing on containerized security tools. If your workflow relies on Docker Compose orchestration or Kubernetes-based scanning infrastructure, you’re stuck with native installations. This matters for CI/CD pipelines that run reconnaissance as part of automated testing—you can’t just add EyeWitness as a service in your docker-compose.yml.
The Chrome/Chromium dependency adds significant disk footprint compared to lightweight headless browsers. This bloat is negligible on workstations but potentially painful on minimal cloud instances or embedded devices used for red team infrastructure. There’s no distributed scanning capability documented—if you need to screenshot tens of thousands of URLs, you’re either waiting for one machine to churn through them or manually splitting the list across multiple hosts and merging outputs yourself. Enterprise-scale reconnaissance may benefit from coordination layers that EyeWitness doesn’t appear to provide natively.
Verdict
Use EyeWitness if you’re performing penetration tests on modern Linux distributions (Kali 2023+, Ubuntu 22.04+) where PEP 668 breaks traditional Python tools, need to process Nmap or Nessus XML output directly into visual reconnaissance reports, or want SQLite-backed resume capability for large URL lists that take hours to complete. The virtual environment architecture delivers genuine cross-platform consistency without container overhead. Skip it if you require Docker/Kubernetes deployment for automated security pipelines (wait for official Docker support to be completed), or need distributed scanning across multiple machines for enterprise-scale reconnaissance. For red teams working on modern operating systems who feed reconnaissance data from network scanners, EyeWitness hits the sweet spot between installation reliability and feature depth.
