Noir: Mining Your Codebase for Shadow APIs Before Attackers Do
Hook
Your API documentation claims 47 endpoints. Your actual codebase exposes 203. Those 156 undocumented routes? Security researchers call them shadow APIs, and they’re often the first breach point.
Context
Traditional API security assumes you know what you’re protecting. You write OpenAPI specs, generate documentation, configure API gateways with explicit route definitions. But reality is messier. Developers add debugging endpoints that never get removed. Legacy routes persist after features are deprecated. Microservices spawn undocumented internal APIs. Authentication bypasses hide in administrative paths. Your security team tests against Swagger files while attackers scan your actual attack surface.
Proxy-based discovery tools like Burp Suite or ZAP’s spider only find endpoints you actually exercise during testing. If you don’t click through the admin panel or trigger that webhook receiver, it stays invisible to your security scans. Documentation-based approaches fail when docs lag behind code—which is always. Noir attacks this problem at the source: it parses your actual codebase to build a comprehensive endpoint inventory, exposing every route defined in your application logic regardless of documentation status or test coverage. Born from OWASP’s security research community, it bridges the gap between what you think you’ve built and what attackers will actually discover.
Technical Insight
Noir operates as a polyglot static analyzer, parsing source trees to extract routing definitions across frameworks. Built in Crystal—a compiled language with Ruby-inspired syntax—it achieves native-code performance while maintaining readable parser implementations. The core architecture revolves around language-specific detectors that pattern-match against framework conventions.
For a Flask application, Noir identifies route decorators and constructs the endpoint map:
# app.py
@app.route('/api/users', methods=['GET', 'POST'])
def users():
return jsonify(users)
@app.route('/api/users/<int:id>', methods=['DELETE'])
def delete_user(id):
return '', 204
@app.route('/admin/debug/sql') # Shadow API - not in docs
def debug_sql():
return render_template('sql_console.html')Running Noir against this codebase extracts all three routes, including the undocumented debug endpoint:
$ noir -u https://api.example.com -b flask app.py
[*] Endpoints detected: 3
[*] Generating output...
GET https://api.example.com/api/users
POST https://api.example.com/api/users
DELETE https://api.example.com/api/users/{id}
GET https://api.example.com/admin/debug/sqlThe output exports to multiple formats. The OpenAPI (Swagger) export creates machine-readable specs for consumption by DAST tools:
# noir_output.yaml
openapi: 3.0.0
paths:
/api/users:
get:
responses:
'200':
description: OK
post:
responses:
'200':
description: OK
/api/users/{id}:
delete:
parameters:
- name: id
in: path
required: true
responses:
'204':
description: No Content
/admin/debug/sql:
get:
responses:
'200':
description: OKThis export becomes the single source of truth for security testing. Feed it to ZAP for automated scanning, import to Burp for manual testing, or integrate with CI/CD pipelines to diff endpoint changes between commits.
Noir’s detector architecture is extensible. Each framework gets a dedicated parser module that understands its routing DSL. The Express.js detector recognizes app.get(), router.post(), and middleware chains. The Spring Boot detector parses @RequestMapping annotations. Django’s detector handles both function-based views and class-based views with URL pattern matching. Currently, Noir supports 20+ frameworks across languages including JavaScript/TypeScript (Express, Fastify, Nest), Python (Flask, Django, FastAPI), Ruby (Rails, Sinatra), Java (Spring), Go (Gin, Echo), and PHP (Laravel).
The AI-powered mode extends coverage beyond native parsers. When you enable it with --llm, Noir sends code snippets to language models to identify routing patterns in unsupported frameworks:
$ noir --llm --provider openai --model gpt-4 custom_framework/This experimental feature trades accuracy for breadth. While native parsers achieve near-100% precision by understanding framework semantics, LLM-based detection might hallucinate endpoints or miss complex routing logic. Use it for reconnaissance when facing unknown codebases, then validate findings manually.
Noir’s Crystal implementation deserves scrutiny. Crystal compiles to native code, delivering performance comparable to Go or Rust while offering Ruby-like syntax that lowers the learning curve for parser development. The compiled binary has no runtime dependencies, making it trivial to integrate into Docker containers or CI systems. However, Crystal’s smaller ecosystem means fewer contributors familiar with the language, potentially slowing feature development compared to Python or JavaScript alternatives.
Gotcha
Static analysis hits walls with dynamic routing. If your application constructs routes at runtime—reading from databases, environment variables, or configuration files—Noir can’t detect them. Consider this Rails pattern:
# Routes loaded from database
DynamicRoute.all.each do |route|
get route.path, to: route.controller
endNoir sees the iteration but can’t predict what DynamicRoute.all returns without executing the code. Similarly, regex-based routing or wildcard patterns may be reported generically rather than expanded to specific paths. Complex middleware chains that conditionally modify routes based on runtime state also elude static analysis.
Framework coverage gaps exist. While Noir handles mainstream frameworks well, proprietary frameworks, heavily customized routing layers, or bleeding-edge frameworks lacking detectors will fall back to LLM analysis or miss routes entirely. The tool’s effectiveness correlates directly with how conventionally your framework structures routing definitions. Metaprogramming-heavy codebases or applications that extensively monkey-patch framework internals will challenge the parsers. The Crystal dependency might friction your workflow—precompiled binaries ease deployment, but contributing patches or debugging requires Crystal proficiency, a language with substantially smaller mindshare than Python or JavaScript.
Verdict
Use if: You’re conducting security assessments against codebases with suspected shadow APIs, integrating endpoint discovery into CI/CD security gates, or feeding DAST tools with comprehensive target lists for white-box testing. Noir excels when you control the source code and need exhaustive attack surface mapping that documentation or proxy-based tools miss. It’s particularly valuable for pentesting teams transitioning from external reconnaissance to code-assisted testing, and for DevSecOps teams establishing endpoint governance across microservice architectures. Skip if: Your application relies heavily on runtime-generated routes, you lack access to source code (stick with traditional black-box discovery), or you’re working with unsupported niche frameworks where the LLM fallback introduces too much noise. For applications with disciplined OpenAPI-first development where specs reliably match implementation, the added complexity may not justify the benefit. Similarly, if Crystal toolchain integration creates organizational friction and your existing SAST tools already extract routing information adequately, the marginal value diminishes.
