Ugly Duckling: How Detectify Built a Zero-Dependency Scanner for Crowdsourced Vulnerabilities

Hook

Most security scanners tout their extensive feature lists and plugin ecosystems. Detectify built Ugly Duckling by deliberately removing features until only the essential scanning logic remained—and that constraint became its greatest strength.

Context

Security companies with crowdsource programs face a peculiar challenge: how do you accept vulnerability detection logic from hundreds of external researchers without creating a security nightmare? You can't give contributors direct access to your production scanner. You can't accept arbitrary code execution. And you certainly can't spend engineering hours debugging every submission.

Detectify Crowdsource needed a submission format that was simple enough for researchers to write and test independently, safe enough to accept from untrusted sources, and structured enough to translate into their internal detection systems. Traditional approaches—accepting curl commands, custom scripts, or Burp Suite extensions—all failed one or more of these requirements. Ugly Duckling emerged as Detectify's answer: a proof-of-concept scanner so minimal it has zero external dependencies, yet flexible enough to express sophisticated vulnerability signatures through pure JSON configuration.

Technical Insight

Ugly Duckling's architecture reveals how much you can accomplish with Go's standard library when you resist feature creep. The entire scanner operates on a simple pipeline: read target URLs from stdin, load JSON detection modules from disk, execute HTTP requests with configurable concurrency, and evaluate responses against match conditions. There's no database, no plugin system, no DSL to learn—just HTTP requests and pattern matching.

A detection module is a JSON file that declares everything about the vulnerability test. Here's what a basic module looks like:

{
  "name": "Apache Struts RCE Detection",
  "request": {
    "method": "GET",
    "path": "/showcase.action",
    "headers": {
      "Content-Type": "application/x-www-form-urlencoded"
    }
  },
  "matches": [
    {
      "pattern": "OGNL Expression",
      "type": "contains"
    },
    {
      "pattern": "^5[0-9]{2}$",
      "type": "regex",
      "target": "status"
    }
  ],
  "matchesRequired": 1,
  "mustNotMatch": [
    {
      "pattern": "404 Not Found",
      "type": "contains"
    }
  ]
}

The genius is in the matching system's flexibility. The matches array defines conditions that indicate a vulnerability—finding specific strings in the response body, matching regex patterns, checking status codes, or examining headers. The matchesRequired field acts as a threshold: set it to 1 for "any of these conditions means vulnerable," or to the array length for "all conditions must be true." The mustNotMatch array provides negative conditions that disqualify a potential hit, preventing false positives.

This design elegantly handles the spectrum of vulnerability detection complexity. Simple cases might check for a single string in the response. Sophisticated modules can require multiple conditions while excluding edge cases, all without writing code. The scanner's Go implementation evaluates these conditions with straightforward conditional logic—no expression parser, no sandboxing overhead.

The concurrency model demonstrates Go's strengths for I/O-bound workloads. Ugly Duckling spawns goroutines for each target-module combination, bounded by a configurable worker pool. Since vulnerability scanning is primarily waiting for HTTP responses, this approach maximizes throughput without the complexity of async/await patterns. The standard library's net/http client handles connection pooling and timeouts, while channels coordinate results back to the main thread for output.

What makes this architecture particularly valuable for Detectify's use case is translatability. Because modules are pure data—no code, no external dependencies, no execution environment assumptions—Detectify's engineers can programmatically convert them to their production scanner's format. A researcher develops and tests their module locally with Ugly Duckling, submits the JSON file, and Detectify translates it into their internal system. The simplicity guarantees compatibility and eliminates an entire class of submission issues.

The zero-dependency constraint also serves a security purpose. Security researchers are notoriously cautious about running third-party tools, and rightfully so. By limiting the codebase to Go's standard library, Ugly Duckling becomes auditable in an afternoon. There's no massive dependency tree to vet, no mysterious native extensions, no supply chain to trust. The entire scanner is a few hundred lines of transparent Go code that compiles to a single static binary.

Gotcha

Ugly Duckling's minimalism is both its strength and its limitation. The scanner lacks features that production security tools consider essential: no authentication mechanisms, no session handling, no cookie jars, no proxy support. If you're testing an endpoint behind OAuth or need to maintain state across multiple requests, Ugly Duckling won't help you. The documentation explicitly states it's alpha-quality software not used in Detectify's production systems—this is a submission and testing tool, not a comprehensive security scanner.

The matching system, while flexible, has edges where it shows its simplicity. Go's regex engine lacks some PCRE features that security researchers might expect, particularly lookahead and lookbehind assertions. Complex response parsing—extracting values from JSON responses to use in subsequent requests, or handling multi-step vulnerability chains—simply isn't supported. If your vulnerability detection requires workflow logic or stateful interactions, you'll need to describe the vulnerability conceptually and let Detectify's internal systems handle the implementation details. For rapid PoC development and straightforward detection cases, these limitations rarely matter. For complex exploit chains, you'll hit the walls quickly.

Verdict

Use Ugly Duckling if you're contributing to Detectify Crowdsource or need a dead-simple scanner for developing vulnerability signatures without wrestling with heavy tooling. Its zero-dependency design and transparent codebase make it perfect for security researchers who value auditability, and the JSON module format is genuinely pleasant for version control and collaboration. It's also an excellent educational resource—reading its source code teaches fundamental scanner architecture without enterprise complexity obscuring the concepts. Skip it if you need production-grade scanning capabilities, authentication support, or complex multi-step vulnerability detection. This isn't Burp Suite or Nuclei with thousands of features; it's intentionally minimal. If your vulnerability requires state management, complex parsing, or workflow logic, you'll outgrow Ugly Duckling immediately. Use it as Detectify intended: a lightweight bridge for submitting detection ideas, not your primary security testing tool.