> your AI agent picks dependencies from memory; give it dated facts — try starlog.dev ↗ vet your agent's deps ↗ vibe-coding is fine. vibe-importing isn’t. — try starlog.dev ↗ vibe-importing isn’t fine ↗ your agent has never seen your private packages — try starlog.dev ↗ facts for private packages ↗ a linter for the dependencies your AI agent picks — try starlog.dev ↗ a linter for agent deps ↗ whois is redacted, cdns mask the rest — get the real operator — try whoisgeni.us ↗ who really runs that domain ↗ domain attribution that shows its work — full evidence chain — try whoisgeni.us ↗ domain intel w/ evidence ↗

← Back to Articles

VulnHunter: Why This AI Scanner Inverts SAST and Forces Exploits to Prove Themselves

[ View on GitHub ]

VulnHunter: Why This AI Scanner Inverts SAST and Forces Exploits to Prove Themselves

Hook

Run VulnHunter twice on the same codebase and you'll get anywhere from 3 to 42 confirmed vulnerabilities. This isn't a bug—it's the honest cost of having an LLM reason like an adversary instead of pattern-match like Semgrep.

Context

Traditional SAST tools start from the sink and work backward: they search for dangerous functions like eval() or execute(), then attempt to trace whether user input can reach them. This produces a flood of false positives because most dangerous functions are actually unreachable from attacker-controlled entry points. You've seen the workflow: Semgrep flags 200 potential SQL injections, your team spends days triaging, and maybe 5 are actually exploitable from HTTP handlers.

VulnHunter, originally built by Capital One's internal security team and now maintained as an open fork by Neal Bridges, inverts this entirely. It starts where an attacker starts—HTTP endpoints, file upload handlers, message queue consumers—and performs forward taint analysis to see what dangerous operations those entry points can reach. More importantly, it implements a falsification pipeline: every finding must survive an explicit attempt to disprove itself before it gets reported. The LLM acts as both prosecutor and defense attorney, and only vulnerabilities that can't be talked out of existence make it into your report. This fork decouples the methodology from Claude Code specifically, making these adversarial reasoning prompts portable to any agent harness that supports markdown skill files.

Technical Insight

VulnHunter's architecture splits into three composable skills that live as plain markdown files with structured phases. The scanner (/vulnhunt) performs reconnaissance to identify attacker-controlled entry points, then launches parallel hunt threads that trace data flow forward toward dangerous sinks. The critical innovation is what happens next: before emitting a finding, it enters an adversarial disproof phase where the LLM actively tries to invalidate its own discovery by searching for sanitization functions, type constraints, or defensive validation that would prevent exploitation.

Here's what a skill invocation looks like in the environment contract:

export VULNHUNT_SKILLS_DIR="/path/to/vulnhunter-skills"
export VULNHUNT_AGENTS_DIR="/path/to/vulnhunter-agent"
export VULNHUNT_MODEL="deepseek-coder-v2:latest"  # Uncensored required
export VULNHUNT_HOST_CMD="ollama run"  # Or vllm, llamafile, etc.

# The scanner skill runs through phases:
# 1. Recon: Find HTTP handlers, file parsers, queue consumers
# 2. Parallel Hunt: Trace taint from each entry point
# 3. Adversarial Disprove: Try to invalidate each finding
# 4. Capability Filter: Confirm exploitability

The forward taint analysis means VulnHunter asks: "Given that I control this HTTP header, what dangerous operations can I trigger?" rather than "Given this SQL execute call, could user input ever reach it?" This eliminates dead code paths that traditional SAST wastes time analyzing. When it finds a path from request.get_param() to os.system(), it doesn't immediately report command injection. Instead, it enters the falsification phase and searches for evidence that the path is actually blocked—shell escaping, allowlist validation, type coercion that would sanitize the input.

The remediation skill (/vulnhunter-fix) implements test-driven security patching. It writes an exploit demonstration first, creates a failing test that proves exploitation works (red), applies a security patch (green), then validates that the exploit is now blocked. This prevents the common pattern where security fixes accidentally break functionality or fail to actually prevent the attack. Here's the conceptual flow:

# Phase 1: Exploit Demo (proves vulnerability is real)
def exploit_csrf_token_bypass():
    response = requests.post(
        "http://target/transfer",
        data={"amount": 1000, "to": "attacker"},
        headers={"Origin": "http://evil.com"}
    )
    assert response.status_code == 200  # No CSRF protection

# Phase 2: Failing Test (red)
def test_csrf_protection():
    # Missing CSRF token should fail
    response = client.post("/transfer", data={...})
    assert response.status_code == 403

# Phase 3: Security Patch (green)
@require_csrf_token
def transfer_handler(request):
    # Implementation with CSRF validation
    pass

# Phase 4: Verify exploit is blocked
def verify_exploit_blocked():
    with pytest.raises(CSRFValidationError):
        exploit_csrf_token_bypass()

The verification skill runs as a separate agent with a restricted tool envelope—no shell access, no network calls—to independently confirm the fix. This separation prevents the hunter from grading its own homework.

Portability comes from treating skills as plain markdown files organized in a directory structure that any agent harness can consume. Instead of building adapters for Langchain, Autogen, CrewAI, and others, VulnHunter exposes environment variables that point to skill directories and model endpoints. Your harness just needs to support loading markdown prompts and executing structured phases. The tradeoff is that the /vulnhunter-fix skill requires a Python helper package and GitHub CLI for PR creation, which breaks the "pure prompt" portability story slightly.

The measured-impact PoC work (still in progress) aims to quantify exploit severity: instead of reporting "SQL injection exists," it would report "27,000 customer records exfiltrable in 3 seconds" or "authentication bypass grants 72-hour session with admin privileges." This shifts from narrative security assessments to engineering metrics—rows leaked, amplification factors, credential-hours exposed. When complete, this makes vulnerability prioritization an evidence-based decision rather than a subjective CVSS score debate.

Gotcha

The 14× variance in findings between identical scans is the elephant in the room. Run VulnHunter on the same repository with the same model and you might get 3 confirmed vulnerabilities one day and 42 the next. This isn't a bug—it's LLM non-determinism propagating through adversarial reasoning. Even with temperature set to zero, the falsification pipeline's multi-step reasoning produces different invalidation chains each run. This makes VulnHunter unsuitable for compliance scenarios where auditors expect deterministic, repeatable results. You can't tell a regulator "we found between 3 and 42 P0 vulnerabilities, depending on which run we cherry-pick."

The requirement for uncensored models is architecturally honest but practically limiting. Commercial API providers like OpenAI and Anthropic implement dual-use cyber safeguards that block exploit generation. Ask Claude or GPT-4 to write a working SQL injection PoC and you'll hit refusal behaviors. This means you need self-hosted infrastructure running open-weight models like DeepSeek Coder, WizardCoder, or Code Llama—and you need the uncensored variants that don't have alignment finetuning. Most organizations already paying for enterprise OpenAI contracts can't just plug VulnHunter into their existing LLM spend; they need separate GPU infrastructure for model hosting. Additionally, VulnHunter lacks the language-specific taint semantics that commercial SAST tools encode as hardcoded rules. It doesn't "know" that Django's parameterized queries prevent SQL injection or that React's JSX auto-escaping blocks XSS—it has to reason about these defenses from first principles based on what the LLM learned during pretraining. A tool like Semgrep can encode "if you see this exact pattern, it's safe" logic; VulnHunter has to rediscover that knowledge through reasoning on every scan.

Verdict

Use if: You're running agent harnesses for security work, have authorization to generate exploits against code you control (red team pre-release audits, not third-party scanning), can host uncensored models locally, and want adversarial reasoning that finds attack paths rather than pattern matches. The falsification pipeline's 55% candidate elimination rate and forward taint analysis from entry points make this qualitatively different from Semgrep/Snyk—you're getting fewer findings, but they're exploitable. Perfect for security teams doing pre-release adversarial reviews where the goal is proving exploitability, not compliance checkbox scanning. Skip if: You need deterministic results for compliance audits, lack infrastructure for self-hosted model hosting, or can't tolerate 14× variance in vulnerability counts between runs. The measured-impact PoC work is incomplete, so right now you're getting structured reasoning about vulnerabilities rather than fully automated runtime exploit validation. This is a tool for augmenting human security review with LLM-powered adversarial thinking, not replacing Semgrep in your CI/CD pipeline. If you need sub-second scans with zero variance and a vendor warranty on findings, stick with CodeQL or Snyk Code.