Inside Nuclei: How YAML Templates Became a Crowdsourced Vulnerability Database

Hook

While commercial scanners spend months integrating new CVE checks, Nuclei templates often appear within hours of disclosure—sometimes before vendor patches exist. The secret? 300+ contributors treating vulnerability detection as code.

Context

Traditional vulnerability scanners face a fundamental tension: comprehensiveness versus speed. Tools like Nessus and Qualys maintain vast databases of vulnerability checks, but their closed development model means weeks or months pass between a CVE announcement and scanner coverage. Open-source alternatives like OpenVAS improved accessibility but inherited the same architectural bottleneck—a small team maintaining thousands of vulnerability signatures.

Nuclei emerged from ProjectDiscovery in 2020 with a radically different thesis: what if vulnerability detection logic lived in simple YAML files that anyone could write, test, and contribute? Instead of maintaining a monolithic scanner codebase, the project separated the engine (Go binary) from the intelligence (YAML templates). This architectural split unleashed community creativity. Security researchers now publish Nuclei templates alongside CVE disclosures. Red teams share reconnaissance techniques as reusable workflows. The template library grew to over 7,000 community-contributed checks covering everything from subdomain takeovers to Kubernetes misconfigurations—a breadth impossible for any single vendor to match.

Technical Insight

Nuclei's power stems from its domain-specific language for expressing vulnerability checks. Each template is a YAML document defining requests, matchers, and extractors. Here's a simplified template that detects exposed Git configuration files:

id: git-config-exposure

info:
  name: Git Config File Exposure
  author: pdteam
  severity: medium
  description: Detects exposed .git/config files

http:
  - method: GET
    path:
      - "{{BaseURL}}/.git/config"

    matchers-condition: and
    matchers:
      - type: word
        words:
          - "[core]"
          - "repositoryformatversion"
        condition: and

      - type: status
        status:
          - 200

      - type: word
        part: header
        words:
          - "application/octet-stream"
          - "text/plain"
        condition: or

This template demonstrates Nuclei's layered matching system. The scanner first issues an HTTP GET request to /.git/config. Then three matchers must all pass (matchers-condition: and): the response contains Git-specific strings, returns HTTP 200, and has appropriate content-type headers. This multi-condition approach dramatically reduces false positives compared to simple signature matching.

Under the hood, Nuclei's execution engine uses a worker pool pattern written in Go. The scanner maintains separate protocol executors—HTTP, DNS, TCP, SSL, and others—each implementing a common interface. When processing a template, Nuclei determines the required protocol executor, prepares the request with variable substitution (like {{BaseURL}}), and dispatches it to worker goroutines. Request clustering groups similar requests to the same host, reducing network overhead:

// Simplified conceptual structure
type Executor interface {
    Execute(template *Template, target string) (*Result, error)
}

type HTTPExecutor struct {
    client *retryablehttp.Client
    rateLimiter *rate.Limiter
}

func (e *HTTPExecutor) Execute(template *Template, target string) (*Result, error) {
    // Variable replacement
    requestURL := strings.ReplaceAll(template.Path, "{{BaseURL}}", target)
    
    // Execute with retry logic
    resp, err := e.client.Get(requestURL)
    if err != nil {
        return nil, err
    }
    
    // Apply matchers
    for _, matcher := range template.Matchers {
        if !matcher.Match(resp) {
            return nil, nil  // No match
        }
    }
    
    return &Result{Matched: true, Template: template.ID}, nil
}

Nuclei's template DSL also supports multi-step workflows, enabling sophisticated attack chain simulation. A template might first extract an API endpoint from JavaScript files, then use that extracted value in a second request to test for authorization bypasses. This workflow capability is expressed through template variables and extractors:

http:
  - method: GET
    path:
      - "{{BaseURL}}/api/config.js"
    
    extractors:
      - type: regex
        name: api_endpoint
        part: body
        regex:
          - 'apiUrl: "([^"]+)"'
        internal: true  # Don't output, use in next request

  - method: GET
    path:
      - "{{api_endpoint}}/admin/users"
    
    matchers:
      - type: status
        status:
          - 200

This chaining transforms Nuclei from a simple scanner into a programmable reconnaissance framework. Templates become reusable security logic that encodes real-world attack patterns, not just regex signatures. The Go implementation provides speed—scanning thousands of hosts concurrently—while YAML keeps the vulnerability logic accessible to security researchers who may not be developers.

The architecture scales through parallelism at multiple levels: concurrent target processing, parallel template execution per target, and clustered requests within templates. Rate limiting and backoff strategies prevent overwhelming targets, while connection pooling and HTTP/2 support optimize network efficiency. This design enables Nuclei to scan entire attack surfaces—hundreds of subdomains, thousands of endpoints—in minutes rather than hours.

Gotcha

Nuclei's community-driven model introduces quality control challenges. Template authors have varying expertise levels, and the maintainers explicitly state they cannot guarantee every template's accuracy or safety. Before running templates against production systems, audit them carefully. Some templates execute potentially destructive actions or generate significant traffic. The severity field provides guidance, but you're responsible for understanding what each template does.

The tool is fundamentally designed as a command-line scanner for periodic execution, not a persistent daemon. The maintainers actively discourage running Nuclei as an always-on service exposed to networks. It lacks authentication mechanisms, session management, and hardening appropriate for server deployments. Teams wanting continuous scanning typically integrate Nuclei into scheduled CI/CD pipelines or orchestration platforms rather than running it standalone. Additionally, the aggressive scanning patterns—especially with large template sets—can trigger WAFs, rate limiters, or even legal concerns if used against targets without authorization. Nuclei provides powerful capabilities but assumes operator judgment about when and where to use them. The rapid release cycle also means breaking changes appear regularly; version pinning and testing in non-production environments before updates is essential for stability.

Verdict

Use Nuclei if: you need rapid coverage of emerging vulnerabilities through community intelligence, you're building security automation into CI/CD pipelines, you scan diverse protocols beyond web applications (DNS, TCP, cloud APIs), or you want to encode custom security checks as code without forking a scanner. It excels at attack surface monitoring and external reconnaissance. Skip if: you need point-and-click GUI workflows (though web interfaces exist, CLI is primary), you require vendor SLAs and guaranteed support without purchasing Enterprise, you need deep stateful application testing with session handling and complex authentication (use dedicated DAST tools), or you're uncomfortable with the legal and technical responsibility of running community-contributed scanning code.