BOTS v3: Splunk's Time-Capsule Dataset for SOC Training That Bypasses Licensing
Hook
BOTS v3 exploits a fascinating licensing loophole: because it ships as finished Splunk indexes rather than raw logs, it never triggers volume-based license calculations. You get enterprise-scale breach telemetry without enterprise costs—if you can resurrect the 2018 app ecosystem.
Context
Training SOC analysts traditionally requires either synthetic data that feels nothing like production, or sanitized enterprise logs that take months to negotiate, clean, and distribute. Splunk's Boss of the SOC (BOTS) competition emerged in 2016 to solve this: give blue teamers a realistic, compromised environment to investigate without requiring live infrastructure, threat actor coordination, or compliance approvals. By version 3 in 2018, BOTS had evolved from simple network captures to full enterprise stacks—AWS CloudTrail, Office 365 audit logs, Sysmon endpoint telemetry, Symantec EPP alerts, network flows from Splunk Stream, osquery tables, and Linux auditd events all timestamped to a single attack scenario.
The key innovation wasn't the data sources themselves, but the distribution format. Instead of shipping gigabytes of raw logs that users must ingest (triggering Splunk's daily indexing volume licenses), BOTS v3 distributes as a 320MB tarball containing Splunk's proprietary index structure: bucket directories with compressed journal.gz files, bloomfilter files, and tsidx time-series indexes already built. Extract it into $SPLUNK_HOME/etc/apps and Splunk immediately recognizes the pre-built 'botsv3' index. No data ingestion happens. No license consumption occurs. It's a frozen storage snapshot made portable, designed for scenarios where spinning up enterprise licenses for training is prohibitive.
Technical Insight
The dataset's architecture is Splunk's index file format itself. When you extract the tarball, you get a directory structure like this:
botsv3/
├── local/
│ └── indexes.conf # Points Splunk to the frozen storage
├── botsv3_frozen/
│ ├── db_1533168000_1533081600_0/
│ │ ├── bucket_info.csv
│ │ ├── Hosts.data
│ │ ├── Sources.data
│ │ ├── SourceTypes.data
│ │ ├── *.tsidx # Time-series indexes
│ │ └── rawdata/
│ │ └── *.journal.gz # Compressed events
│ └── db_1533254400_1533168000_1/
│ └── ...
The indexes.conf configuration tells Splunk to treat these frozen buckets as searchable:
[botsv3]
homePath = $SPLUNK_DB/botsv3/db
coldPath = $SPLUNK_DB/botsv3/colddb
thawedPath = $SPLUNK_DB/botsv3/thaweddb
frozenTimePeriodInSecs = 188697600
When Splunk starts, it scans frozen directories and rebuilds the index manifest without re-parsing events. Field extractions happen at search time using the 24 required apps/add-ons. This dependency tree is where things get fragile. The dataset assumes exact versions of TA-microsoft-sysmon 6.0.0, Splunk_SA_CIM 4.12.0, SA-AccessProtection 3.2.2, and 21 others from 2018. Each provides sourcetype-specific field extractions and Common Information Model (CIM) mappings.
For example, Sysmon process creation events rely on TA-microsoft-sysmon's transforms.conf to extract parent processes, command lines, and hashes:
[sysmon-process-create]
REGEX = <Data Name='ParentCommandLine'>([^<]+)</Data>
FORMAT = parent_process::$1
WRITE_META = true
Searching the dataset for lateral movement patterns looks like:
index=botsv3 sourcetype="XmlWinEventLog:Microsoft-Windows-Sysmon/Operational"
EventCode=3
dest_ip=192.168.250.*
dest_port=445
| stats dc(src_ip) as unique_sources by dest_ip
| where unique_sources > 5
This search breaks if the Sysmon add-on version differs from 6.0.0 because field extraction logic evolved. Version 8.0.0 changed how Image and ParentImage paths are normalized, causing dest_ip and src_ip to fail mapping from the CIM Network Traffic data model.
The 107 sourcetypes span seven log categories representing realistic heterogeneity. AWS CloudTrail (aws:cloudtrail) shows API calls like AssumeRole and GetSessionToken that attackers use post-compromise. Symantec Endpoint Protection (symantec:ep:risk:file) contains detection alerts with hash IOCs. Splunk Stream (stream:http, stream:dns) captures network conversations at the wire level. Office 365 (ms:o365:management) logs email access and SharePoint exfiltration. This breadth lets analysts practice pivots: find a suspicious PowerShell execution in Sysmon, pull the script hash, search Symantec EPP for other hosts with the file, check Stream DNS for C2 beacons, review CloudTrail for credential abuse in AWS.
The pre-indexed approach has a hidden cost: opacity. You cannot re-parse events with different field extractions or test how alternative Sysmon configurations would affect visibility. The journal.gz files contain raw events, but extracting and re-ingesting them defeats the licensing bypass advantage and requires processing 12GB of uncompressed data. The dataset is read-only by design—a deliberate tradeoff between accessibility and flexibility.
Gotcha
The exact version pinning makes deployment on modern Splunk installations a nightmare. Splunk Enterprise 9.x uses Python 3, restructured REST API endpoints, and updated CIM schemas that break assumptions from the 7.1 era. Apps from 2018 frequently reference deprecated SPL commands (like typelearner) or use authentication methods no longer supported. You'll spend hours in _internal logs debugging compatibility issues, often requiring manual edits to app .conf files to comment out obsolete stanzas.
The attack telemetry itself is five years stale. Techniques like DCShadow, Kerberoasting variations, and modern ransomware deployment patterns aren't represented. The malware samples use C2 protocols and infrastructure from 2018—domain generation algorithms, HTTP beaconing patterns, and TLS certificates that contemporary threat intelligence feeds don't recognize. If you're validating current detection rules or researching emerging TTPs, this dataset teaches you to hunt ghosts. Additionally, the single-scenario design means no baseline data exists. You can't measure false positive rates, test detection rules against normal operations, or understand what benign PowerShell activity looks like in this environment. Every search assumes you're hunting—there's no 'clean' period for comparison.
Verdict
Use if: You're training junior SOC analysts on Splunk query syntax and investigation workflows without budget for enterprise licenses, running university courses where students need hands-on SIEM experience with realistic multi-source data, or practicing blue team CTF scenarios where the goal is learning correlation techniques rather than discovering novel attacks. The licensing bypass and sourcetype diversity remain unmatched for education.
Skip if: You're building production detection rules (the telemetry is obsolete and lacks baseline data for tuning false positives), researching current adversary techniques (2018 TTPs don't reflect modern cloud-native attacks or evasion methods), running Splunk Enterprise 9.x without tolerance for dependency archaeology (the 24 required apps will break in creative ways), or need a reproducible lab environment where you can modify attack variables and regenerate telemetry. For those use cases, invest in Mordor datasets with raw event formats or build adversary emulation infrastructure with Caldera.