NiuDao: Building a Security Testing Memory Layer with AI and Yaklang
Hook
Most AI pentesting tools forget everything after each scan. NiuDao treats your past exploits as training data, distilling successful attack patterns into reusable tactics that guide future assessments.
Context
Security testing tools have a fragmentation problem. A typical engagement requires orchestrating nmap, sqlmap, nuclei, Burp Suite, and dozens of other specialized tools—each with its own syntax, output format, and installation quirks. ARTEX attempted to solve this by building an AI coordinator that invokes these tools on demand, but it inherited the same operational headaches: dependency hell, version conflicts, and the cognitive overhead of managing twenty different binaries.
NiuDao takes a different approach by replacing ARTEX's distributed tool ecosystem with a single backend: Yaklang, a unified security testing engine with a gRPC interface. But the real innovation isn't tool consolidation—it's the evidence-settling loop. Unlike ChatGPT wrappers that generate pentesting suggestions and hope for the best, NiuDao persists every scan result, traffic capture, and tactical decision in PostgreSQL with full traceability. When an attack workflow succeeds, the system distills it into a 'tactic' (战法)—a structured record of what worked, under what conditions, and what evidence proved it. Future scans inherit these tactics as few-shot examples, turning your pentesting history into a knowledge base that guides LLM reasoning.
Technical Insight
The architecture is deliberately three-tiered to separate concerns that typical pentest tools conflate. PostgreSQL 16 handles persistence—not just storing findings, but maintaining an auditable chain from task definition → tool invocation → evidence collection → tactical extraction. The Go backend (1.26.3) houses the agent orchestration engine, which doesn't directly execute security tools. Instead, it communicates with a separately installed Yaklang instance via gRPC, treating Yak as a stateless executor for scans, exploits, and traffic analysis.
Here's how a typical workflow looks in the codebase. The agent receives a task like "enumerate subdomains for target.com and test for SQL injection." Instead of spawning subprocess calls to dig and sqlmap, it constructs a Yaklang invocation:
// Simplified from internal/agent/executor.go
func (e *YakExecutor) RunScan(ctx context.Context, task *Task) (*Evidence, error) {
// Build Yaklang script from task parameters
script := fmt.Sprintf(`
subdomain = str.ParseStringToHosts("%s")~
for host in subdomain {
res, err = poc.HTTP("%s", poc.https(true))~
if err == nil { poc.Save(res) }
}
`, task.Target, task.Target)
// Invoke via gRPC bridge
stream, err := e.yakClient.Exec(ctx, &ypb.ExecRequest{
Script: script,
})
// Collect outputs and traffic logs
var artifacts []Artifact
for {
resp, err := stream.Recv()
if err == io.EOF { break }
artifacts = append(artifacts, Artifact{
Type: resp.Type,
Data: resp.Raw,
Timestamp: time.Now(),
})
}
// Persist to evidence table with provenance metadata
return e.db.StoreEvidence(task.ID, artifacts)
}
The key insight is in the evidence-settling loop. After Yaklang returns results, the system doesn't just display them—it prompts the LLM to correlate findings with stored tactics. If previous tasks found SQLi in login forms via time-based payloads, that tactic gets injected into the agent's context:
// Simplified from internal/agent/tactics.go
func (a *Agent) EnrichPrompt(task *Task) (string, error) {
// Query similar successful tasks
tactics, err := a.db.FindTactics(task.Target, task.Type)
if err != nil { return "", err }
var examples strings.Builder
for _, t := range tactics {
examples.WriteString(fmt.Sprintf(
"Previous success: %s on %s using %s (evidence: %s)\n",
t.Outcome, t.Target, t.Method, t.EvidenceHash,
))
}
return fmt.Sprintf(`
You are a security tester. Target: %s
Available tools: Yaklang engine (subdomain enum, vuln scanning, exploit dev)
Relevant past tactics:
%s
Plan your approach and specify Yaklang scripts to run.
`, task.Target, examples.String()), nil
}
This creates a feedback loop where successful patterns accumulate. The LLM doesn't learn in the traditional ML sense—there's no gradient descent or model updates. Instead, tactical distillation works like expert system rule extraction: when a task completes successfully, the agent generates a natural language summary of the approach ("time-based SQLi payloads against /api/login?user= parameter"), stores it with hash references to the raw evidence (traffic captures, tool outputs), and surfaces it during similar future tasks.
The Next.js frontend exposes this workflow through a task dashboard where operators can review evidence chains. Unlike autonomous pentesting tools that run to completion, NiuDao surfaces intermediate findings and waits for human approval before escalating—pausing after reconnaissance to confirm targets before launching exploitation phases. This async model is crucial: security testing isn't a blocking operation, and evidence accumulation often spans hours or days as targets respond to different payloads.
The Yaklang integration happens through MCP (Model Context Protocol) bindings, not embedded execution. You install Yak separately and configure the gRPC endpoint in NiuDao's environment. This separation means you can upgrade Yaklang independently or even swap it for another gRPC-compatible engine—though the codebase currently hardcodes Yak-specific script syntax, so portability is theoretical. The trade-off is operational: one engine to secure, one interface to monitor, versus the sprawl of managing dozens of tool installations with inconsistent logging and auth mechanisms.
Gotcha
The README explicitly warns that "not all scenarios are end-to-end validated," and there's no CI/CD pipeline to verify Yaklang integration at scale. This matters because the entire system depends on Yak's gRPC bridge remaining stable—if a Yaklang update breaks protocol compatibility or changes script semantics, NiuDao's evidence collection and tactical distillation silently fail. You're committing to maintaining two codebases (NiuDao and Yaklang) with no upstream guarantees about interface stability.
Tactical distillation also inherits LLM hallucination risks in a subtle way. The system doesn't verify that a "successful" tactic actually caused the outcome—it correlates evidence with task completion and lets the LLM infer causality. If a scan finds nothing but the model generates a false positive summary ("discovered SQLi via payload X"), that bogus tactic gets stored and influences future runs. There's no ground truth validation, no A/B testing of tactics, and no anomaly detection to flag suspiciously high-confidence extractions. The licensing situation is equally problematic: third-party code in the skills/ directory lacks clear copyright metadata, and the README explicitly blocks public releases until this is resolved. If you fork or deploy this commercially, you're accepting undefined legal exposure.
Verdict
Use if: You already run Yaklang in your security operations and want AI-driven orchestration without managing twenty different tool installations. The tactical distillation system offers genuine value over ChatGPT wrappers—you get historical pattern matching and provable evidence chains instead of purely generative suggestions. This is ideal for red teams willing to invest engineering time validating the Yak bridge and debugging evidence-settling logic in exchange for a unified workflow. Skip if: You need production-ready stability, regulatory-grade audit trails, or can't dedicate resources to maintaining dual dependencies (NiuDao + Yaklang). The unresolved skills/ licensing makes this unsuitable for organizations with software supply chain compliance requirements. For turnkey solutions with mature RBAC and validated tool integrations, stick with Metasploit Framework or commercial platforms. If you want AI assistance without Yaklang lock-in, evaluate PentestGPT—it's lighter weight and avoids the gRPC coupling, though you lose the evidence workflows that make NiuDao compelling for teams who can tolerate its experimental status.