> your AI agent picks dependencies from memory; give it dated facts — try starlog.dev ↗ vet your agent's deps ↗ vibe-coding is fine. vibe-importing isn’t. — try starlog.dev ↗ vibe-importing isn’t fine ↗ your agent has never seen your private packages — try starlog.dev ↗ facts for private packages ↗ a linter for the dependencies your AI agent picks — try starlog.dev ↗ a linter for agent deps ↗ whois is redacted, cdns mask the rest — get the real operator — try whoisgeni.us ↗ who really runs that domain ↗ domain attribution that shows its work — full evidence chain — try whoisgeni.us ↗ domain intel w/ evidence ↗

← Back to Articles

ARTEX: Building Autonomous Penetration Testing Agents with Human-in-the-Loop Controls

[ View on GitHub ]

ARTEX: Building Autonomous Penetration Testing Agents with Human-in-the-Loop Controls

Hook

What if your penetration testing tool could autonomously enumerate attack surfaces, but physically couldn't fire exploits without you explicitly pressing 'approve' in a web UI?

Context

Traditional penetration testing tools fall into two camps: fully manual frameworks like Burp Suite that require human operators to click through every request, or fully automated scanners like Nuclei that blast through templates without context. Both approaches have fatal flaws. Manual testing doesn't scale—a single web application with hundreds of endpoints can take days to enumerate properly. Automated scanners lack adaptability—they can't chain discoveries together ('I found an admin panel, so now I should look for default credentials') without pre-scripted playbooks.

Large language models promised a middle path: AI agents that could reason about targets and adaptively choose next steps. But early implementations like PentestGPT exposed a new problem—giving an LLM unrestricted shell access is terrifying. One hallucinated command could delete production databases or trigger incident response teams. ARTEX solves this by implementing a synchronous approval queue: AI agents can plan and execute reconnaissance freely, but any operation flagged as sensitive (POST requests, exploit attempts, scope violations) blocks until a human reviews and approves it. The architecture treats AI as a tireless junior pentester who needs supervision, not a fully autonomous attacker.

Technical Insight

View Pending Ops

Approve/Reject

Decompose Tasks

Poll Status Every 5s

Invoke Skills

Submit Sensitive Ops

HTTP Requests

Record & Replay

Read/Write State

Query Context

Expose Operations

Scoped Auth

Resume on Approval

Next.js Frontend

Static Export

Planner Agent

LLM Orchestrator

Worker Agents

Concurrent Pool

Skills Plugin System

Go/Python Scripts

pending_operations

Approval Queue

PostgreSQL

State & Schema

HTTP Proxy :8788

Traffic Recording

MCP Endpoint

External Agent API

System architecture — auto-generated

ARTEX's core innovation is its polling-based approval system backed by PostgreSQL. When an agent wants to execute a sensitive operation, it doesn't call a webhook or emit an event—it writes a row to the pending_operations table and enters a polling loop. Here's the worker-side flow:

// Agent detects a POST request to /admin/delete
opID := submitPendingOperation(ctx, Operation{
    Type: "http_request",
    Method: "POST",
    URL: "https://target.com/admin/delete",
    Body: `{"user_id": 123}`,
    Reason: "Attempting to test authorization bypass",
})

// Block until approval
for {
    status, err := checkOperationStatus(ctx, opID)
    if err != nil {
        return err
    }
    if status == "approved" {
        // Execute the operation
        resp, err := httpClient.Post(url, contentType, body)
        break
    } else if status == "rejected" {
        return fmt.Errorf("operation rejected by operator")
    }
    time.Sleep(5 * time.Second) // Poll every 5s
}

Meanwhile, the web UI queries pending_operations where status = 'pending' and renders an approval interface showing the full request context. Operators click 'Approve' or 'Reject', which updates the row's status. The polling agent picks up the change and resumes.

This design survives restarts—if the system crashes mid-approval, the pending operation remains in the database. When ARTEX restarts, workers resume polling from where they left off. It's inefficient compared to websockets (wasted database queries during polling), but eliminates callback hell and race conditions.

The skill system extends this agent framework with filesystem plugins. Rather than compiling all reconnaissance tools into the binary, ARTEX loads Python scripts from ./skills/ at runtime:

#!/usr/bin/env python3
# skills/subdomain_enum.py
#DESCRIPTION: Enumerate subdomains using DNS queries and certificate transparency logs
#PARAM: domain (string, required) - Target root domain
#PARAM: wordlist (string, optional) - Path to subdomain wordlist

import sys
import json
import dns.resolver
import requests

def enumerate_subdomains(domain, wordlist=None):
    results = []
    # Check certificate transparency
    ct_response = requests.get(f"https://crt.sh/?q=%.{domain}&output=json")
    for entry in ct_response.json():
        results.append(entry['name_value'])
    
    # Brute force if wordlist provided
    if wordlist:
        with open(wordlist) as f:
            for line in f:
                subdomain = f"{line.strip()}.{domain}"
                try:
                    dns.resolver.resolve(subdomain, 'A')
                    results.append(subdomain)
                except:
                    pass
    
    return {"subdomains": list(set(results))}

if __name__ == "__main__":
    params = json.loads(sys.argv[1])
    output = enumerate_subdomains(**params)
    print(json.dumps(output))

The skill loader parses the metadata comments (#DESCRIPTION, #PARAM) to generate OpenAI function schemas. When the planner agent decides 'I need to enumerate subdomains for target.com', the LLM runtime sees:

{
  "name": "subdomain_enum",
  "description": "Enumerate subdomains using DNS queries and certificate transparency logs",
  "parameters": {
    "type": "object",
    "properties": {
      "domain": {"type": "string", "description": "Target root domain"},
      "wordlist": {"type": "string", "description": "Path to subdomain wordlist"}
    },
    "required": ["domain"]
  }
}

The agent invokes it, ARTEX executes python3 skills/subdomain_enum.py '{"domain": "target.com"}', captures stdout, parses the JSON response, and feeds results back to the LLM for next-step reasoning. Practitioners can drop custom tools into ./skills/ without touching Go code—the system auto-discovers and registers them.

Scope enforcement uses PostgreSQL's CIDR types to prevent agents from attacking out-of-scope targets:

CREATE TABLE company_assets (
    id SERIAL PRIMARY KEY,
    company_id INT NOT NULL,
    ip_range CIDR,
    domain_pattern TEXT,
    ports INT[]
);

CREATE INDEX idx_ip_range ON company_assets USING GIST (ip_range inet_ops);

-- Before spawning a worker for 192.168.1.50:443
SELECT EXISTS (
    SELECT 1 FROM company_assets
    WHERE company_id = $1
      AND ip_range >>= '192.168.1.50'::inet
      AND 443 = ANY(ports)
);

If the query returns false, the operation enters the approval queue with a 'scope violation' flag. This provides hard boundaries even when LLMs hallucinate targets or misinterpret instructions.

The frontend embeds into the Go binary as a static export. During build, next build && next export generates ./out/ with pre-rendered HTML. The Go binary embeds this via:

//go:build embedui
//go:embed all:out
var uiAssets embed.FS

func serveUI() http.Handler {
    stripped, _ := fs.Sub(uiAssets, "out")
    return http.FileServer(http.FS(stripped))
}

This produces a single portable executable with no Node.js runtime dependency. Updates download new binaries, validate SHA256 checksums, stage as artex.new, then exit with code 42. The wrapper script start.sh detects this, swaps binaries, and restarts—achieving zero-downtime updates without container orchestration.

Gotcha

The polling-based approval system trades simplicity for efficiency. Every blocked agent hammers the database every 5 seconds checking for status updates. With 20 concurrent workers each waiting on approval, you're generating 240 queries per minute just for polling—before any actual work happens. This doesn't scale to hundreds of workers without connection pool tuning and query optimization. A websocket-based approach or PostgreSQL's LISTEN/NOTIFY would eliminate the polling overhead, but would require managing connection lifecycles and handling reconnection logic.

Skill execution has zero sandboxing beyond filesystem isolation. Python scripts run with full access to environment variables (which include database credentials and LLM API keys), the network, and the ./skills and ./data directories. A malicious skill—or worse, an LLM-generated payload that gets auto-saved and executed—could exfiltrate secrets, pivot to internal networks, or corrupt the database. Production deployments need external sandboxing (run ARTEX inside a VM with network segmentation) or skill auditing (review every .py file before execution). The system assumes you trust your skill library completely, which is dangerous when LLMs can hallucinate malicious tool calls.

Verdict

Use ARTEX if: You're a red team operator who wants to delegate tedious recon ('find all login pages for Company X') to AI while maintaining veto power over destructive actions. You have custom reconnaissance scripts that you want to integrate without rewriting them in Go. You need audit trails for every action taken during an engagement. You're comfortable running self-hosted infrastructure and managing PostgreSQL.

Skip ARTEX if: You need fully autonomous exploitation without human intervention—the approval queue is a deliberate brake, not a bug. Your security policies prohibit sending target data to third-party LLM APIs (the system requires OpenAI/Anthropic/etc.). You're building a multi-tenant SaaS platform—the lack of skill sandboxing makes this a single-operator tool only. You want compatibility with existing MCP clients that expect stdio transport—ARTEX only implements Streamable HTTP. You need the mature exploit library and post-exploitation modules that Metasploit provides—ARTEX excels at reconnaissance, not payload delivery.