Interlace: The Thread Pool Hack That Parallelizes Your Single-Threaded Security Tools

Hook

Most penetration testers waste 80% of their engagement time waiting for single-threaded tools to scan targets sequentially, even though the network latency makes parallelization trivial—if you know how to orchestrate it.

Context

The security tooling ecosystem has a threading problem. Tools like nmap, nikto, and dirb were built in an era when scanning a /24 network meant 254 targets, not the thousands of endpoints modern bug bounty programs expose. These tools were designed for thoroughness, not speed, and adding native threading to decades-old C codebases is often impractical or breaks assumptions about execution order.

Meanwhile, penetration testers and red teamers need to run the same reconnaissance commands against hundreds or thousands of targets during the initial enumeration phase. Running "nikto -h example.com" sequentially against 500 web servers turns a 30-second task per target into a 4-hour marathon of waiting. GNU Parallel exists, but it doesn't understand security-specific concepts like CIDR notation, and constructing safe parallel commands with proper output management requires bash expertise most security practitioners don't have. Interlace emerged from this gap: a Python wrapper that treats your existing single-threaded tools as black boxes while adding parallelization, target expansion, and workflow orchestration on top.

Technical Insight

At its core, Interlace is a command templating engine with a thread pool executor. You provide a command template with variables, a target list, and a thread count—Interlace handles expansion, substitution, and parallel execution. The simplest case demonstrates the power:

interlace -tL targets.txt -threads 10 -c "nmap -p- _target_ -oN _output_/_target_.nmap"

This command reads targets.txt (which can contain hostnames, IPs, CIDR ranges like 192.168.1.0/24, or even ranges like 10.0.0.1-50), spawns 10 threads, and runs nmap against each target. The _target_ variable gets replaced with each expanded target, while _output_ references the directory specified with -o. It's deceptively simple, but the implementation reveals thoughtful design choices.

The target expansion engine is where Interlace differentiates itself from generic parallelization tools. Feed it "192.168.1.0/24,scanme.nmap.org,10.0.0.1-10" and it expands this into 265 individual targets before queuing work. The parser handles ipaddress module integration for CIDR blocks, regex matching for ranges, and comma-delimited lists—all the input formats security tools consume but rarely produce. This means you can pipe output from one tool directly into Interlace without transformation scripts.

The variable substitution system includes safety features often overlooked in shell scripting. The _safe-target_ variable performs character sanitization to prevent command injection when targets come from untrusted sources. If a target contains shell metacharacters, _safe-target_ replaces them with underscores while _target_ preserves the original. This matters when dealing with discovered subdomains that might include backticks or semicolons:

interlace -tL discovered-subdomains.txt -threads 5 \
  -c "curl -H 'Host: _target_' https://example.com/ -o results/_safe-target_.html"

The real architectural sophistication appears in the blocker/block system, which enables per-target execution dependencies while maintaining parallelism across targets. Imagine you need to create target-specific output directories before running scans. The naive approach creates directories serially, then runs scans—killing parallelism. Interlace's solution uses execution phases:

interlace -tL targets.txt -threads 20 \
  -c "mkdir -p _output_/_target_ && _blocker_" \
  -c "nmap _target_ -oA _output_/_target_/nmap && _block_"

The first command with _blocker_ executes for all targets, creating directories in parallel. The _blocker_ keyword marks this command as a blocking operation that must complete before proceeding. The second command with _block_ waits until all blockers finish, then executes in parallel across all targets. This creates a two-phase execution graph: setup phase (parallelized), then scanning phase (parallelized), with a synchronization barrier between them.

This pattern extends to complex workflows. You might run a fast port scanner as a blocker, parse the results to identify open ports, then launch service-specific enumeration tools in the block phase. The threading model uses Python's concurrent.futures.ThreadPoolExecutor under the hood, which means you're bounded by GIL for CPU-intensive work but that's irrelevant here—network I/O dominates security tooling workloads.

Interlace also provides protocol-aware features through the -p flag for ports. Combine it with _port_ variables and you get nested parallelization:

interlace -tL hosts.txt -p 80,443,8080,8443 -threads 50 \
  -c "nikto -h _target_:_port_ -o _output_/_target_-_port_.txt"

This expands hosts × ports, creating individual tasks for each combination. With 10 hosts and 4 ports, you get 40 parallel tasks. The thread pool ensures you don't overwhelm your network or trigger rate limiting by spawning thousands of processes—a problem with naive xargs implementations.

Gotcha

The biggest operational limitation is error handling and visibility. When you're running 500 parallel scans and 50 of them fail due to timeouts, DNS errors, or target unavailability, Interlace provides minimal feedback. There's no built-in retry mechanism, no centralized error log aggregating failures, and no way to query which targets succeeded versus failed without manually checking output files. The timeout flag -t exists, but timed-out commands are simply killed—their partial output may be lost, and you won't get a summary of what needs re-running.

This becomes painful during long-running engagements. If you're scanning 1,000 targets overnight and 100 randomly fail due to transient network issues, you'll discover this by noticing missing output files the next morning, then manually diffing your target list against results to identify gaps. Production-grade tools would maintain state, log structured errors, and offer idempotent retry logic.

The lack of rate limiting or adaptive throttling is another practical limitation. The -threads flag provides coarse-grained control, but there's no way to say "make 10 requests per second maximum" or "back off if you hit HTTP 429 responses." When testing against production systems or APIs with documented rate limits, you're forced to artificially lower thread counts and hope you stay under the threshold. Tools like httpx or nuclei have built-in rate limiters that respect target infrastructure—Interlace treats all commands as black boxes and can't implement protocol-specific throttling.

Output management reveals philosophical differences too. Interlace encourages filesystem-based organization (create directories per target, write files with templated names) rather than structured data formats. This works well for manual review during penetration tests but makes automated result aggregation harder. If you want to feed 500 nmap XML files into a database or generate executive reports, you're writing custom parsers. Contrast this with modern scanning frameworks that output JSON, support webhook callbacks, or integrate with databases directly.

Verdict

Use if: You're conducting penetration tests or bug bounty reconnaissance where you need to run legacy tools (nmap, nikto, dirb, sqlmap) against hundreds of targets quickly, and those tools lack native threading. The blocker/block primitives make Interlace particularly valuable for complex multi-stage scanning workflows where later stages depend on earlier results—like using masscan for port discovery then launching service-specific tools against discovered ports. It's also ideal when you're working with diverse target input formats (CIDR, ranges, lists) that need normalization. Skip if: You need production-grade reliability features like automatic retries, structured error logging, or state management for long-running campaigns. Also skip if rate limiting and respectful scanning are critical—you'll need tools with protocol-aware throttling. If your tools already have good native parallelization (like modern nuclei or httpx), adding another orchestration layer just complicates debugging. Finally, avoid Interlace if you're building automated pipelines that need structured output for downstream processing rather than human-reviewed filesystem artifacts.