OSINT Framework: The Wikipedia of Intelligence Gathering Tools That Lives in a Single JSON File
Hook
A 700-line JSON file with 11,000+ GitHub stars has become the de facto reference for intelligence gathering tools—no database, no backend, no authentication. Just a tree of links rendered in vanilla JavaScript.
Context
Before OSINT Framework emerged in 2015, security researchers, investigators, and journalists maintained personal bookmarks or internal wikis of intelligence gathering tools. If you needed to find someone's social media profiles, trace an IP address, or investigate domain ownership, you'd either know the right tool already or spend hours searching forums and blog posts. The OSINT (Open Source Intelligence) community had knowledge scattered across DefCon presentations, Twitter threads, and dated blog posts.
Justin Seitz (lockfale) recognized that the OSINT community needed a structured, maintainable catalog rather than another blog post that would become outdated in months. The solution wasn't a complex platform—it was a hierarchical taxonomy of tools with just enough metadata to be useful. The framework organizes resources by investigation type: start with what you have (a username, email, domain, IP address), navigate the tree, and discover tools that can extract intelligence from that starting point. It's essentially a decision tree for intelligence gathering, packaged as a static website that anyone can fork, modify, and self-host.
Technical Insight
The entire OSINT Framework architecture revolves around osint.json (originally arf.json), a nested JSON structure that defines categories, subcategories, tools, and metadata. Each node in the tree can be either a category container or a leaf node representing an actual tool. The data structure is elegantly simple:
{
"Username": {
"children": {
"Username Search Engines": {
"children": {
"Namechk": {
"T": "(T) - Checks username availability across 100+ social networks",
"link": "https://namechk.com",
"flag": "free",
"opsec": "passive"
},
"Sherlock": {
"T": "(T) - Hunt down social media accounts by username",
"link": "https://github.com/sherlock-project/sherlock",
"flag": "free",
"opsec": "active"
}
}
}
}
}
}
The metadata schema includes several practical fields: flag indicates pricing (free/paid/freemium), opsec denotes whether the tool is passive (target unaware) or active (leaves traces), and the T field provides a description. This operational security distinction is critical—passive tools query public databases without alerting targets, while active tools might trigger logging or notifications.
The frontend rendering logic recursively walks this JSON structure to build an interactive DOM tree. The core function in osint.js dynamically creates nested <ul> and <li> elements with click handlers for expand/collapse behavior:
function buildTree(obj, parentElement) {
Object.keys(obj).forEach(key => {
if (key === 'children') {
buildTree(obj[key], parentElement);
} else if (typeof obj[key] === 'object' && obj[key].children) {
// Category node
const li = document.createElement('li');
li.className = 'category';
const span = document.createElement('span');
span.textContent = key;
span.onclick = () => li.classList.toggle('expanded');
li.appendChild(span);
const ul = document.createElement('ul');
buildTree(obj[key].children, ul);
li.appendChild(ul);
parentElement.appendChild(li);
} else {
// Leaf node (actual tool)
const li = document.createElement('li');
li.className = 'tool';
const link = document.createElement('a');
link.href = obj[key].link;
link.textContent = key + ' - ' + obj[key].T;
li.appendChild(link);
parentElement.appendChild(li);
}
});
}
This architecture choice—static JSON consumed by client-side JavaScript—has significant implications. There's no server processing, no database queries, no authentication layer. The entire site can be served from GitHub Pages, cached aggressively, and works perfectly offline once loaded. Contributors don't need to understand REST APIs or database schemas; they just edit JSON and submit pull requests.
The taxonomy itself represents years of community knowledge distillation. Top-level categories map to common investigation starting points: Username, Email Address, Domain Name, IP Address, Social Networks, and specialized areas like Dark Web, Public Records, and Geospatial. Each branch organizes tools by function rather than by provider, so you'll find "Email to Name" and "Email to Breach Data" as separate subcategories under Email Address.
What makes this design particularly clever is how it handles tool proliferation. As new OSINT tools emerge (and they emerge constantly), contributors can add entries without restructuring the entire taxonomy. The JSON schema is forgiving—you can add custom metadata fields without breaking existing parsers. Some entries include notes about API requirements, registration needs, or legal restrictions:
"Hunter.io": {
"T": "(T) - Email finder using domain search with free tier limits",
"link": "https://hunter.io",
"flag": "freemium",
"note": "Requires account. 25 free searches/month",
"opsec": "passive"
}
The framework also includes a search function that filters the tree in real-time, implemented with simple string matching against category names and tool descriptions. It's not sophisticated—no fuzzy matching, no ranking algorithm—but it's fast and predictable because everything's already loaded in memory.
Gotcha
The OSINT Framework's greatest strength—its simplicity—is also its most significant limitation. The project catalogs tools but provides zero quality assurance. A tool listed in 2016 might have shut down, changed its pricing model, or degraded into uselessness, but it remains in the JSON file until someone notices and submits a PR. With hundreds of external services, link rot is endemic. You'll frequently encounter dead links, services that now require payment despite the 'free' flag, or tools that simply don't work as described.
The framework also lacks any concept of tool effectiveness or community ratings. Sherlock and Namechk both appear under username searches with equal prominence, but Sherlock is actively maintained with 40k+ GitHub stars while Namechk's reliability varies wildly. You won't find guidance on which tools are actually trustworthy, which have the best coverage, or which respect privacy. The flat presentation treats a robust, open-source tool the same as someone's weekend project that hasn't been updated in years. For newcomers to OSINT, this creates a paradox of choice without the context needed to make informed decisions. You're handed a map with 200 destinations but no indication of which roads are paved versus washed out. Experienced practitioners already know which tools to trust, so they use the framework for discovery rather than recommendation. But if you're building your first OSINT workflow, you'll waste considerable time testing tools that don't deliver, troubleshooting defunct services, and trying to figure out why a 'passive' tool is actually making direct connections to target infrastructure.
Verdict
Use if: You're building custom intelligence gathering workflows and need comprehensive awareness of available tools across different investigation types, you're an OSINT practitioner who wants a quick reference for finding resources in unfamiliar domains (geospatial, cryptocurrency, dark web), or you're teaching intelligence gathering and need a structured curriculum of tools to demonstrate. The framework excels as a discovery layer and educational resource. Skip if: You need vetted, actively maintained tool recommendations with reliability assessments, you're looking for an integrated OSINT platform that actually executes searches rather than linking to external tools, or you want workflow guidance and methodology rather than just a catalog. Consider SpiderFoot or Maltego if you need automated execution, or IntelTechniques.com for higher-quality curation. The OSINT Framework is perfect for researchers who know how to evaluate tools themselves but want comprehensive coverage of what exists.
