CocoIndex: The React Reconciliation Model for AI Agent Context

Hook

Refactoring your PDF chunking strategy forces you to re-embed your entire document corpus. Or it did, until someone built a dataflow engine that hashes transformation function bytecode.

Context

Production RAG systems face a problem that traditional ETL tools weren't designed to solve: context drift. Your AI agent needs fresh embeddings when documentation updates, but re-processing thousands of documents on every git commit burns through API quotas and adds minutes of latency. Airflow and Prefect can schedule incremental runs, but they track changes at the pipeline level, not the row level. Change your chunking logic from 512 to 1024 tokens? Both orchestrators see a code change and re-run everything, even though 90% of your documents haven't been modified.

The deeper issue is that existing tools conflate two orthogonal concerns: data changes and code changes. Traditional CDC (Change Data Capture) systems like Debezium detect when source rows mutate but ignore transformation logic updates. Analytics orchestrators like dbt handle schema evolution but use partition-based incremental models that can't detect which specific records need recomputation when you refactor a Python function. CocoIndex emerged from this gap, built specifically for teams running long-horizon AI agents where context freshness directly impacts output quality and stale embeddings cause hallucinations.

Technical Insight

CocoIndex's architecture centers on persistent-state-driven incremental computation, a model borrowed from React's virtual DOM reconciliation but applied to data pipelines. When you decorate a Python function with @coco.fn(memo=True), the Rust core wraps it in a content-addressed memoization layer that hashes both input data and the function's bytecode. Each execution stores a mapping of hash(input_bytes) + hash(function_code) → output_value in a persistent cache. On subsequent runs, the engine computes hashes for incoming data and current code, checking the cache before executing. If either hash changes, the function re-runs; otherwise, it returns the cached result.

Here's what a basic incremental pipeline looks like:

import cocoindex as coco
from openai import OpenAI

@coco.source
def pdf_documents():
    return coco.mount_directory("./docs", pattern="*.pdf")

@coco.fn(memo=True)
async def extract_text(pdf_path: str) -> str:
    # Content-addressed: only re-runs if PDF bytes OR this function changes
    return await parse_pdf(pdf_path)

@coco.fn(memo=True)
async def chunk_text(text: str) -> list[str]:
    # Changing chunk_size here only re-processes affected documents
    chunk_size = 1024
    return [text[i:i+chunk_size] for i in range(0, len(text), chunk_size)]

@coco.fn(memo=True)
async def embed_chunk(chunk: str) -> list[float]:
    client = OpenAI()
    response = await client.embeddings.create(
        input=chunk,
        model="text-embedding-3-small"
    )
    return response.data[0].embedding

@coco.target
async def vector_index(chunks_with_embeddings):
    coco.declare_vector_index(
        name="docs_index",
        records=chunks_with_embeddings,
        embedding_field="embedding"
    )

The killer feature surfaces when you modify chunk_size from 1024 to 512. Traditional pipelines re-embed everything because the transformation code changed. CocoIndex computes hash(extract_text.bytecode) and sees it's unchanged, so extract_text results stay cached. Then it computes hash(chunk_text.bytecode), detects the modification, and re-executes chunk_text only for cached text outputs. Finally, embed_chunk receives new chunk strings (different hashes) and re-calls the OpenAI API only for affected chunks. Unmodified chunks from unchanged documents? Served from cache.

The engine builds a reactive dependency graph where each node tracks input provenance. When you call declare_vector_index, CocoIndex doesn't imperatively write to your vector database. Instead, it reconciles declared state against current state, computing a minimal diff of inserts/updates/deletes. This mirrors React's virtual DOM — you declare what the index should contain, and the engine figures out the minimal mutations.

Per-row lineage tracking enables bidirectional queries. Want to know which source PDF contributed bytes to a specific embedding vector? The Rust core maintains a lineage pointer from each output record back to its input hashes. This solves the RAG explainability problem: trace a hallucinated response back to the exact document chunk and source file. Conversely, when a PDF updates, the engine queries the lineage graph to find all downstream embeddings that need invalidation.

The Python async interface is a leaky abstraction from Rust's async runtime. CocoIndex embeds Tokio (Rust's async executor) via PyO3 bindings, so your transformation functions must be async def to integrate with the work-stealing scheduler. The engine parallelizes function execution across CPU cores without requiring Spark clusters, but you pay for this with async/await boilerplate. Synchronous PDF parsers need wrapping in asyncio.to_thread() or executor pools, adding friction compared to synchronous orchestrators like Airflow.

The memoization layer uses content-addressed storage, meaning identical transformations across different pipelines share cached results. If five separate flows all call the same chunk_text function on the same input, the expensive computation executes once and all five pipelines read the cached output. This deduplication is automatic and based on hash equality, not manual cache key management.

Gotcha

CocoIndex's single-process execution model hits a ceiling around low terabytes of data. The Rust runtime parallelizes across cores using work-stealing, but you're bounded by vertical scaling — one machine's RAM and CPU. Processing a billion-row table requires either beefier hardware or manual sharding across multiple CocoIndex instances, which defeats the declarative simplicity. Apache Spark and Flink transparently distribute work across clusters; CocoIndex makes you handle horizontal scale yourself.

Memoization storage grows unbounded without manual intervention. Every code version and input variant creates a cache entry. Refactor your chunking function ten times during development? You've now got ten cached versions of every document's chunks, consuming disk. There's no documented TTL, LRU eviction, or automatic garbage collection. At scale, the persistent cache will bloat unless you build custom pruning logic, and the project docs don't yet address cache lifecycle management. Additionally, storing full lineage metadata per-row adds overhead — a billion-row table with five-hop transformations incurs non-trivial metadata storage compared to systems like dbt that only track partition-level lineage. The trade-off buys you fine-grained provenance queries, but it's not free.

Verdict

Use CocoIndex if you're running production RAG systems or AI agents where context staleness degrades output quality, and you have medium-scale data (gigabytes to low terabytes) with frequent incremental updates. The code-aware caching is genuinely novel and will cut your embedding API costs by an order of magnitude when you're iterating on chunking strategies or updating subsets of documents. It excels for living data sources like codebases, Slack archives, or documentation sites where full re-indexing is prohibitively expensive. Skip if you're building batch-oriented analytics (dbt is simpler and more mature), need transparent horizontal scaling across clusters (use Spark or Flink), or have append-only data where full re-computation is cheap. Also skip if your transformations rely heavily on synchronous libraries — the async-everywhere requirement adds too much friction. The sweet spot is exactly what they advertise: keeping long-horizon agent context synchronized with evolving data sources without burning compute on redundant work.