TOON: The Data Format That Makes LLMs 40% Cheaper to Feed
Hook
Every JSON object you send to GPT-4 costs you 40% more tokens than necessary. A new serialization format called TOON is rewriting the rules of prompt engineering by treating LLMs as the primary audience, not humans.
Context
The rise of large language models has created an unexpected cost center: token consumption. Every API call to GPT-4, Claude, or Llama charges per token, and developers quickly discovered that their carefully crafted prompts were hemorrhaging tokens through verbose JSON serialization. When you pass a 500-record employee dataset to an LLM for analysis, JSON’s repetitive structure—where every object repeats its keys—becomes expensive fast. A 100KB JSON payload might cost $0.30 in GPT-4 tokens, and if you’re making thousands of calls daily, you’re looking at real money.
Token-Oriented Object Notation (TOON) emerged from this economic reality. Unlike JSON, which was designed for human readability and JavaScript compatibility, TOON treats token efficiency as a first-class concern. It borrows YAML’s indentation for nested objects and introduces CSV-style tabular layouts for arrays of uniform objects—the most common pattern in LLM prompts. The format explicitly declares schemas using [N] for array lengths and {field1,field2} for table headers, giving LLMs structural guardrails that improve parsing accuracy while slashing token counts. It’s not trying to replace JSON everywhere; it’s optimizing the specific use case of feeding structured data to language models.
Technical Insight
TOON’s architecture centers on two core serialization strategies: indented key-value pairs for objects and tabular representation for uniform arrays. Here’s how a typical JSON structure transforms:
// Original JSON (127 tokens)
const employees = {
"company": "Acme Corp",
"employees": [
{"id": 1, "name": "Alice", "role": "Engineer", "salary": 120000},
{"id": 2, "name": "Bob", "role": "Designer", "salary": 95000},
{"id": 3, "name": "Carol", "role": "Manager", "salary": 135000}
]
};
// TOON representation (76 tokens)
company: Acme Corp
employees: [3]
{id,name,role,salary}
1, Alice, Engineer, 120000
2, Bob, Designer, 95000
3, Carol, Manager, 135000The token savings come from three architectural choices. First, TOON eliminates quote marks around keys and uses colons for simple values, reducing visual noise. Second, the [3] array length declaration tells the LLM exactly how many rows to expect, providing an error-checking mechanism that JSON lacks. Third, the tabular format with explicit field headers {id,name,role,salary} means each field name appears once instead of three times, cutting repetition dramatically.
The TypeScript SDK implements this through a two-phase serializer. The first phase analyzes your JSON structure to detect “tabular eligibility”—arrays where every object shares identical keys. The second phase chooses representation strategies per-node:
import { toToon, fromToon } from '@toon-format/toon';
// Serialize JSON to TOON
const toonString = toToon(employees, {
indent: 2,
tabularThreshold: 2 // Require at least 2 uniform objects
});
// Parse TOON back to JSON (lossless)
const reconstructed = fromToon(toonString);
// reconstructed is deeply equal to original employees objectThe library handles edge cases carefully. Nested arrays stay nested in tabular format—each cell can contain sub-arrays or sub-objects, with the parser tracking indentation levels to determine scope. String values containing commas or newlines get quoted automatically, maintaining CSV’s escaping rules. The format also degrades gracefully: if an array contains even one object with mismatched keys, that entire array falls back to indented representation rather than breaking parsing.
What makes TOON particularly effective for LLMs is the explicit schema declaration. When you tell GPT-4 to “extract the top 3 salaries from this data,” the [3] and {id,name,role,salary} markers act as structural hints. The benchmarking data shows this improves retrieval accuracy from 70% (JSON) to 74% (TOON) on complex queries, because the model doesn’t have to infer structure—it’s declared upfront. This is especially valuable for smaller models like Llama-7B, where parsing ambiguity significantly impacts accuracy.
The SDK also includes a CLI tool for batch conversion and a benchmarking framework that tests both token efficiency and retrieval accuracy across different data shapes. The benchmarks reveal TOON’s sweet spot: data with high “tabular eligibility”—think API responses, database query results, or configuration files with repeated structures. On the reference benchmark dataset (mix of e-commerce orders, user profiles, and time-series data), TOON averages 40% fewer tokens than JSON. But drill into the numbers, and you’ll find that performance varies wildly: 55% savings for pure tabular data, 25% for moderately nested structures, and occasionally negative savings (more tokens than JSON) for deeply nested or highly variable schemas.
Gotcha
TOON’s efficiency collapses when your data doesn’t fit its tabular worldview. If you’re working with deeply nested configuration files, graph structures, or arrays where every object has different keys, TOON becomes actively worse than JSON. The format has to fall back to indented representation for non-uniform arrays, and the overhead of array length markers [N] and failed tabular detection can actually increase token count by 5-10% compared to compact JSON. Before integrating TOON, run your actual production data through the benchmarking tool—don’t assume you’ll hit that 40% average.
The second limitation is ecosystem maturity. TOON is a 22,000-star project, but tooling remains thin. There’s no native support in popular LLM frameworks like LangChain or Semantic Kernel, meaning you’ll write conversion glue code at every prompt boundary. Editor support exists for VS Code, but syntax highlighting and validation are rudimentary compared to JSON’s decades of tooling investment. If your team is already struggling with prompt engineering complexity, adding a custom serialization format increases cognitive load. You’re also betting on TOON’s longevity—if the format doesn’t achieve critical adoption mass, you might be stuck maintaining conversion layers for years. And while the spec claims lossless bidirectional conversion, edge cases exist: JSON’s numeric precision handling differs from TOON’s string-based tabular cells, potentially causing floating-point drift in round-trip conversions.
Verdict
Use if: You’re making high-volume LLM API calls with structured data that contains uniform arrays (employee records, transaction logs, sensor data, API responses), token costs are a measurable line item in your budget, and you can afford the integration overhead of adding conversion steps to your prompt pipeline. The 40% token savings and improved parsing accuracy are real, and they compound quickly at scale. Also use it if you’re working with smaller open-source models where the explicit schema declarations meaningfully improve accuracy.
Skip if: Your data is deeply nested or highly variable (complex configuration files, graph structures, polymorphic objects), you’re optimizing for latency over token cost (some quantized models process JSON faster despite higher token counts), your team lacks bandwidth to learn a new format and maintain conversion tooling, or you’re working with pure flat tables where CSV is simpler and more universally supported. Also skip if your LLM usage is low-volume where token savings won’t offset the engineering time to integrate TOON.
