Sigma Specification: The Rosetta Stone for Cross-Platform Security Detection
Hook
Every time you switch SIEM vendors, your entire detection engineering library becomes worthless. Sigma exists to fix that—by treating security detections like infrastructure-as-code.
Context
Before Sigma, detection engineering was a vendor-locked nightmare. Your team writes 500 Splunk SPL detection rules over three years. Then your company acquires another division running Elastic, or migrates to Microsoft Sentinel for cost reasons. Suddenly, you're facing months of manual translation work, rewriting each detection from scratch in a new query language. The logic is identical—'alert when PowerShell executes with encoded commands'—but the syntax, field names, and aggregation operators are completely different.
This problem compounds in open-source threat intelligence communities. A researcher discovers a novel ransomware technique and wants to share detection logic with the security community. Should they publish in Splunk's SPL? Elastic's KQL? SQL for QRadar? Each choice excludes 80% of potential users. The result: detection knowledge gets published as prose descriptions in blog posts, forcing each organization to manually implement the logic in their specific SIEM. Sigma emerged from the threat detection community around 2017 to solve this: a vendor-neutral specification for expressing detection rules as portable YAML files that compile to any backend query language.
Technical Insight
At its core, Sigma is a specification-first project defining detection rules through three architectural layers: rule structure, field normalization taxonomy, and modifier pipelines. The genius is in how these layers compose to bridge fundamentally incompatible log schemas.
A Sigma rule looks deceptively simple:
title: Encoded PowerShell Command Execution
status: stable
logsource:
category: process_creation
product: windows
detection:
selection:
Image|endswith: '\\powershell.exe'
CommandLine|contains:
- '-enc'
- '-encodedcommand'
condition: selection
falsepositives:
- Legitimate admin scripts
level: medium
But this 15-line YAML compiles to radically different backend queries. For Splunk, it generates: (Image="*\\powershell.exe") AND (CommandLine="*-enc*" OR CommandLine="*-encodedcommand*"). For Elastic, it becomes KQL: Image.keyword:*\\powershell.exe AND (CommandLine:*-enc* OR CommandLine:*-encodedcommand*). For Microsoft Sentinel's native schema, it maps Image to ProcessName and CommandLine to CommandLine with different escaping rules.
The logsource selector is where architectural elegance emerges. Instead of hardcoding 'Sysmon Event ID 1' or 'Windows Security Event 4688', Sigma uses abstract categories. The three-tier hierarchy—product (windows/linux/aws), service (sysmon/cloudtrail), and category (process_creation/network_connection)—lets backend converters map to specific log sources. A rule targeting product: windows, category: process_creation works whether your environment uses Sysmon, Windows Security Logging, EDR telemetry from CrowdStrike, or Carbon Black audit logs. The conversion tool (typically pySigma) handles the mapping through configuration files:
# Backend mapping configuration
logsources:
windows-process_creation:
product: windows
category: process_creation
conditions:
EventID: 4688 # Windows Security
rewrite:
Image: NewProcessName
CommandLine: CommandLine
The modifier system is where Sigma's power for real-world detection becomes apparent. Attackers encode, obfuscate, and transform payloads. Detecting base64-encoded PowerShell commands in UTF16-LE encoding (common in Windows) requires chaining transformations:
detection:
selection:
CommandLine|base64offset|contains:
- 'Invoke-Mimikatz'
- 'DumpCreds'
condition: selection
The base64offset modifier tells the backend: 'decode this field trying all three base64 offset variations (0, 1, 2 byte shifts), then check if decoded content contains these strings'. In native SPL or KQL, you'd write custom functions or regex nightmares. Sigma backends generate the correct vendor-specific implementation automatically.
Modifier composition gets more sophisticated for multi-stage obfuscation:
detection:
selection:
ScriptBlockText|re:
- '\$[a-zA-Z_][a-zA-Z0-9_]*\s*=\s*\[System\.Text\.Encoding\]::Unicode\.GetString\(\[System\.Convert\]::FromBase64String\('
condition: selection
This regex modifier matches PowerShell that's programmatically decoding base64-encoded Unicode strings—a common dropper pattern. Different SIEM platforms have wildly different regex engines (PCRE, RE2, .NET). Sigma backends handle the dialect translation.
The specification's correlation rules solve the stateful detection problem. Simple Sigma rules are atomic—one event matches or doesn't. But real attacks span multiple events: a process spawns, makes network connections, writes files. Correlation rules express temporal relationships:
title: Suspicious Process Network Activity
status: experimental
correlation:
type: event_count
rules:
- process_creation_suspicious
- network_connection_external
group-by:
- Image
timespan: 5m
condition:
gte: 1
This says: 'if a process matches the suspicious_creation rule, then makes an external network connection within 5 minutes, alert'. The backend compiler generates vendor-specific join logic—Splunk's transaction command, Elastic's EQL sequences, or Sentinel's join operator. The abstraction leaks here—backends have different performance characteristics for temporal joins, and complex correlation rules may not compile efficiently everywhere.
The field taxonomy appendix is 1,500+ lines mapping canonical field names to vendor schemas. CommandLine might map to cmdline, ProcessCommandLine, command_line, or CommandLine depending on the log source. This normalization is manually curated, and it's explicitly incomplete. The specification acknowledges perfect normalization is impossible—AWS CloudTrail's eventName has no equivalent in Windows Event Logs. But 80% coverage enables a rule marketplace.
Gotcha
Sigma's lowest-common-denominator approach means you can't express sophisticated detections that leverage vendor-specific features. Want to use Splunk's machine learning toolkit for anomaly scoring? Can't express it in Sigma. Need Elastic's EQL sequence syntax to detect 'process A spawns B, which writes file C, then B makes network connection to D'? Sigma's correlation rules support event counting and grouping, but not complex sequence logic with variable binding. The correlation timespan model assumes batch-style windowing—you define a 5-minute or 1-hour window. Real-time streaming backends like Kafka Streams or Apache Flink struggle to map this efficiently, often requiring buffering that increases latency.
The modifier composition system is powerful but fragile. Modifiers apply left-to-right as a pipeline: fieldname|base64|utf16le|contains means 'base64 decode, then UTF16-LE decode, then check contains'. Reversing the order produces different results, and there's no type system to catch mistakes. If you write fieldname|contains|base64, the backend will try to base64-decode the string 'true' or 'false' (the boolean result of contains), which is nonsense. Validation only happens at compile time when converting to backend queries, not at rule authoring time. Field taxonomy conflicts exist because governance is community-consensus-based rather than technical enforcement. You'll find rules using both CommandLine and command_line, User and username, with no canonical answer—different backend configurations make different choices.
Verdict
Use if: You're running multi-SIEM environments (hybrid cloud with Splunk on-prem and Sentinel in Azure), contributing to or consuming open-source detection content, or building a detection engineering program that needs to survive vendor migrations. Sigma is the only mature abstraction layer for cross-platform detection logic, and the pySigma tooling plus SigmaHQ's 3,000+ rule library provide immediate value. Also use it if you want detection-as-code workflows—version-controlled YAML rules that compile and deploy via CI/CD pipelines work better than clicking through SIEM UIs. Skip if: You're standardized on a single SIEM and need advanced vendor-specific features. Splunk's machine learning, Elastic's graph analysis, or Sentinel's entity behavior analytics can't be expressed in Sigma's portable syntax. Also skip for complex behavioral detections requiring statistical aggregations beyond counting (standard deviation, percentile-based thresholds, time-series forecasting). Sigma targets signature-based threat detection, not UEBA or anomaly detection use cases where vendor-specific algorithms provide real defensive value.