Seval: A Rust CLI That Makes LLM-Powered Offensive Security Actually Safe
Hook
Most LLM-powered security tools give you two choices: manually approve every nmap scan, or let GPT-4 run 'rm -rf /' on your reconnaissance server. Seval's approval mode system is the first architecture I've seen that actually solves this.
Context
The offensive security community has been cautiously experimenting with LLM assistants since GPT-4's release, but the results have been underwhelming. Tools like PentestGPT require you to manually copy-paste tool output back to the LLM, defeating the purpose of automation. On the other end, fully autonomous agents are terrifyingly dangerous when they have shell access—one hallucination during a red team engagement could wipe your reconnaissance data or accidentally DDoS the client's infrastructure.
Seval, written in Rust by Hari Nagarajan, attempts to thread this needle with a permission system that offers granular control over what an LLM can execute autonomously. It's a terminal-based assistant designed specifically for reconnaissance and exploitation workflows, with built-in tools for shell commands, file operations, web scraping, and search. The architecture centers on session persistence—conversations are stored in SQLite with project-scoped memory, meaning the tool builds institutional knowledge about specific targets across multiple sessions without manual knowledge base management.
Technical Insight
Seval's core innovation is its four-tier approval system that gates every tool invocation before execution. The modes are: 'plan' (show me what you'd do), 'default' (ask permission for each tool), 'auto-edit' (execute read-only tools automatically, ask for writes), and 'yolo' (full autonomy). This maps cleanly to real penetration testing workflows—you start in 'plan' mode during reconnaissance scoping, switch to 'auto-edit' when you're confident in the target list, and only use 'yolo' for repetitive tasks you've already validated.
The tool registry uses a trait-based dispatch system where each of the 10 built-in tools (shell, read_file, write_file, web_fetch, web_search, list_dir, get_cwd, list_tools, summarize, update_mem) returns structured output that gets serialized back into the LLM's context. Here's what the flow looks like for a typical reconnaissance task:
// Simplified tool execution flow
match approval_mode {
ApprovalMode::Plan => {
// LLM generates tool calls, user sees the plan
display_tool_plan(&tool_calls);
return; // Don't execute
},
ApprovalMode::Default => {
for tool_call in tool_calls {
if !prompt_user_approval(&tool_call) {
continue;
}
let result = execute_tool(&tool_call)?;
append_to_context(result);
}
},
ApprovalMode::AutoEdit => {
for tool_call in tool_calls {
if is_destructive(&tool_call) && !prompt_user_approval(&tool_call) {
continue;
}
let result = execute_tool(&tool_call)?;
append_to_context(result);
}
},
ApprovalMode::Yolo => {
// Execute everything, pray to your model provider
for tool_call in tool_calls {
execute_tool(&tool_call)?;
}
}
}
The session management architecture is clever for offensive workflows. Every conversation is automatically persisted to SQLite with a project-scoped memory system—when you run Seval in a directory, it loads memories associated with that filesystem path. This means if you're working on reconnaissance for 'target-corp.com', you can close your session, come back three days later, and the LLM immediately has context about previous scans, discovered subdomains, and open ports without you manually feeding it history.
Context window management addresses a real problem in long reconnaissance sessions: you hit model token limits fast when you're feeding in nmap output, directory listings, and web scrapes. Seval implements automatic summarization—when you approach the context limit, older messages get compressed through the LLM itself. The TUI shows a color-coded sidebar indicating current token usage (green < 50%, yellow 50-80%, red > 80%), so operators know when they're about to lose detailed history. This is lossy, but it's the right tradeoff for reconnaissance where you care more about recent findings than the exact command you ran two hours ago.
The multi-provider abstraction layer deserves attention. Seval supports both AWS Bedrock and OpenRouter through a unified interface, which matters because model selection is surprisingly important in offensive work. Claude 3.5 is excellent at analyzing complex network configurations but sometimes refuses to generate exploit code due to safety training. In contrast, uncensored Llama models on OpenRouter don't have those guardrails but hallucinate more on technical analysis. The ability to switch providers mid-conversation means you can use Claude for recon and Llama for exploitation without losing session state.
The shell tool is both the most powerful and most dangerous primitive. It exposes raw command execution, which is the only way to integrate with existing offensive tooling—nmap, ffuf, sqlmap, gobuster, nuclei. The approval modes are what make this safe enough to use. In 'default' mode, you see the exact command before it runs. In 'auto-edit' mode, read-only commands like 'ls' or 'cat' execute automatically, but anything with write potential ('rm', piped commands, 'wget') requires approval. This dramatically reduces approval fatigue while maintaining safety.
Gotcha
The tool registry is hardcoded at 10 primitives with no plugin architecture. If you need framework-specific integrations—Metasploit RPC, Burp Suite API calls, custom reconnaissance scripts—you're forking and recompiling Rust code. This is a deal-breaker for teams with established offensive toolchains. Most mature red team operations have dozens of custom scripts and integrations; Seval can't absorb any of them without code changes.
Tool execution is entirely synchronous, which kills performance on slow operations. When the LLM decides to run an nmap scan against 254 hosts, your entire agent loop blocks for minutes. No background jobs, no concurrent task scheduling, no way to queue multiple reconnaissance operations and process results as they complete. This architectural choice simplifies state management but means Seval can't match the throughput of hand-rolled scripts with parallelized enumeration. The web scraping tools (web_fetch and web_search) also lack headless browser support, so JavaScript-heavy targets—basically the entire modern web—are invisible to reconnaissance. You get raw HTML, which is useless for single-page applications where all the interesting content is client-side rendered. Finally, context summarization is one-way and lossy. Once messages get compressed, you can't retrieve the original conversation. This matters for compliance and reporting—red team engagements often require detailed logs of exactly what commands were run and what output was observed, not AI-generated summaries.
Verdict
Use if: You're a solo penetration tester or bug bounty hunter who wants LLM assistance during reconnaissance without the risk of autonomous disasters, you're comfortable with terminal interfaces and Rust tooling, and your targets are traditional server-side applications where raw HTTP requests are sufficient for reconnaissance. The approval mode system is genuinely innovative and makes this the safest autonomous LLM tool I've tested for offensive work. Skip if: You need team collaboration beyond JSON session exports, you require integration with enterprise offensive platforms like Metasploit or Cobalt Strike, your targets are modern JavaScript-heavy web applications, or you need audit trails that meet compliance requirements. The lack of plugin architecture and synchronous execution model make this a personal productivity tool rather than a platform for serious red team operations.