> your AI agent picks dependencies from memory; give it dated facts — try starlog.dev ↗ vet your agent's deps ↗ vibe-coding is fine. vibe-importing isn’t. — try starlog.dev ↗ vibe-importing isn’t fine ↗ your agent has never seen your private packages — try starlog.dev ↗ facts for private packages ↗ a linter for the dependencies your AI agent picks — try starlog.dev ↗ a linter for agent deps ↗ whois is redacted, cdns mask the rest — get the real operator — try whoisgeni.us ↗ who really runs that domain ↗ domain attribution that shows its work — full evidence chain — try whoisgeni.us ↗ domain intel w/ evidence ↗

← Back to Articles

ARTEX: Building an AI Pentest Agent with External Binary Skills and Session Approval Gates

[ View on GitHub ]

ARTEX: Building an AI Pentest Agent with External Binary Skills and Session Approval Gates

Hook

Most AI agent frameworks run tools as in-process functions. ARTEX shells out to standalone Go binaries instead—enabling hot-reload, sandboxing, and a skill system that won't crash your entire pentest if nuclei segfaults.

Context

Autonomous penetration testing has a trust problem. Give an LLM agent unfettered access to sqlmap and burpsuite, and you're one hallucinated command away from dropping production databases. Most AI security tools solve this with rigid guardrails—predefined attack chains, read-only reconnaissance, or human-driven execution where the LLM just suggests next steps. But this defeats the autonomy promise: you wanted an agent that explores unknown attack surfaces while you sleep, not a chatbot that needs hand-holding.

ARTEX tackles this with an approval workflow architecture: agents propose dangerous operations, execution pauses, a human reviews the plan, and work resumes only after explicit consent. It's autonomous until it matters, then collaborative. The system couples this with a skill system where security tools live as external Go binaries rather than imported libraries, and a browser extension that captures authenticated sessions by recording your manual login flow. The result is a framework that balances automated exploration with human oversight, targeting small security teams running repeated pentests against familiar environments where the cost of review is lower than the cost of a missed vulnerability.

Technical Insight

Persistence

Skill Execution

Session Management

POST browsing sessions

captured traffic

serves static

creates

delegates subtasks

updates

LLM context

JSON-RPC stdin

dangerous ops

pending review

tool output

session data

task-scoped data

schema.sql on startup

Browser Extension

HTTP Proxy :8788

Go API Server

Embedded Next.js

Planner Agent

Work Agents x3

Conversation State

LLM Provider

./skills/ Binaries

Approval Layer

PostgreSQL

SQLite

System architecture — auto-generated

ARTEX's architecture centers on sessions containing multiple work agents coordinated by a planner. Each session tracks conversation state with an LLM provider (Anthropic Claude, OpenAI, or Ollama), a token budget, and a task scope with associated targets. The planner delegates subtasks—reconnaissance, exploitation, verification—to concurrent work agents (default 3) that execute tool calls through the skill system.

The skill system is the most architecturally interesting piece. Rather than registering tools as in-process function calls like LangChain or AutoGPT, ARTEX treats each security tool as a standalone binary in ./skills/. When an agent wants to run nuclei, the Go backend doesn't import a nuclei library—it shell-executes ./skills/nuclei with JSON arguments via stdin:

// Simplified skill execution from internal agent logic
func (a *Agent) ExecuteSkill(skillName string, params map[string]interface{}) (string, error) {
    skillPath := filepath.Join("./skills", skillName)
    cmd := exec.Command(skillPath)
    
    // JSON-RPC style input via stdin
    input, _ := json.Marshal(params)
    cmd.Stdin = bytes.NewReader(input)
    
    output, err := cmd.CombinedOutput()
    if err != nil {
        return "", fmt.Errorf("skill %s failed: %w", skillName, err)
    }
    
    // Agent receives raw output to parse with LLM
    return string(output), nil
}

This external binary approach has trade-offs. On the positive side: skills can be written in any language (though the repo is Go-heavy), hot-reloaded without restarting the core service, and sandboxed via OS process isolation. A crashing nuclei binary doesn't take down the agent orchestrator. You can version skills independently and swap implementations (replace gobuster with ffuf) without touching core code.

The downside is orchestration overhead. Every skill invocation spawns a process, serializes JSON, and waits for the binary to exit. Long-running scans like nmap can't stream partial results—the agent blocks for 30 minutes until the full scan completes, then parses the output. There's no mechanism for agents to react mid-scan when early results would change strategy. The agent also receives raw text output rather than structured data, relying on the LLM to parse tool output correctly. An LLM interpreting nmap's human-readable format can hallucinate open ports or miss edge cases that a proper XML parser wouldn't.

The approval workflow solves the autonomy trust problem with an interception layer. When an agent proposes a risky action—SQL injection testing, brute force attacks, exploit execution—the session pauses and writes the pending operation to PostgreSQL with status awaiting_approval. A frontend UI surfaces these for human review:

// Approval check before executing dangerous skills
if requiresApproval(skillName) {
    approvalID := createApprovalRequest(sessionID, skillName, params)
    
    // Polling loop - agent sleeps until human decides
    for {
        status := checkApprovalStatus(approvalID)
        if status == "approved" {
            break
        } else if status == "rejected" {
            return "", errors.New("operation rejected by human reviewer")
        }
        time.Sleep(5 * time.Second)
    }
}

This is pragmatic for small teams doing sprint-cycle pentests, but becomes a bottleneck at scale. Three work agents finding 15 SQLi candidates require 15 manual approval clicks. There's no batch approval, policy engine ("auto-approve read-only scans"), or risk scoring to prioritize reviews. Every risky action pauses all work agents in that session, serializing what should be parallel exploration.

The browser extension for traffic capture is genuinely clever. Traditional pentesting proxies like Burp intercept HTTPS with a CA certificate, but this breaks modern web apps with certificate pinning and struggles with CAPTCHA, OAuth, or 2FA flows. ARTEX's browser extension lets you manually navigate authenticated sessions (complete the CAPTCHA, solve the 2FA challenge) while it records every request in the background. When you're done, it POSTs the captured traffic as JSON to /api/traffic/ingest:

// Browser extension capture (simplified)
chrome.webRequest.onBeforeRequest.addListener(
  (details) => {
    capturedRequests.push({
      url: details.url,
      method: details.method,
      headers: details.requestHeaders,
      body: details.requestBody,
      timestamp: Date.now()
    });
  },
  { urls: ["<all_urls>"] },
  ["requestBody", "requestHeaders"]
);

// Send to ARTEX backend when user finishes auth flow
function uploadSession() {
  fetch('http://localhost:8788/api/traffic/ingest', {
    method: 'POST',
    body: JSON.stringify(capturedRequests)
  });
}

This effectively transfers human authentication capabilities to agent context. The agent gets a full HAR-style session dump including cookies, auth tokens, and POST bodies from your manual interaction. It can then replay requests, fuzz parameters, or extract API endpoints without dealing with login complexity. The limitation is secret handling: captured tokens are stored in PostgreSQL with truncation at 256KB but no mention of encryption at rest, and the extension sends data over HTTP to localhost (no TLS on the internal API).

Deployment is a single binary thanks to Go's embed.FS. The Next.js frontend is statically exported and embedded at compile time:

//go:embed frontend/out/*
var staticFiles embed.FS

func ServeUI() http.Handler {
    return http.FileServer(http.FS(staticFiles))
}

This eliminates versioning mismatches and deployment dependencies—no separate nginx config or Node server. The downside is development ergonomics: hot-reload requires running a separate npm run dev server for frontend work, and rebuilding the binary for every UI change during integration testing.

Gotcha

The asset graph uses client-side D3 force-directed layouts with no virtualization or hierarchical clustering. This works beautifully for demos with 50 assets but chokes on realistic enterprise environments. A /16 network with thousands of services and interconnections will freeze your browser tab. The README shows screenshots of clean, sparse graphs that don't reflect production complexity.

Schema migrations run idempotently via IF NOT EXISTS guards in schema.sql executed on every startup. This is pragmatic for single-author projects but brittle at scale. There's no proper migration framework (Flyway, Liquibase), so down migrations or data transformations require manual SQL surgery. Adding a column is easy; changing a column type while preserving data requires custom scripts outside the framework. The retest workflow with dedicated retester agents tracks remediation cycles (Still reproducible / Fixed / Cannot confirm), which is genuinely useful for sprint-based security work. But the LLM nondeterminism in tool output parsing makes reruns flaky. The same SQL injection might be "confirmed exploitable" in run one and "cannot confirm" in run two if the agent's interpretation of sqlmap output varies. There's no validation layer or deterministic parser to ensure consistency across retests.

Verdict

Use if: You're a small security team (2-5 people) running repeated pentests against the same environments across sprint cycles, you need human-in-the-loop review as a feature not a bug, and you're testing authenticated web apps where the browser extension traffic capture shines. The retest tracking and approval workflow make this excellent for vulnerability verification work where you're confirming fixes, not discovering 0days. Skip if: You're auditing cloud-scale infrastructure (the asset graph won't render 10,000+ assets), operating fully autonomously (the approval bottleneck kills parallel progress), need batch processing without human intervention (every risky action requires a click), or require reproducible compliance reports (LLM nondeterminism makes reruns inconsistent). Also skip if you're doing red team ops where speed and stealth matter—the external binary skill overhead and approval workflow serialization are too slow for time-sensitive adversary simulation.