> your AI agent picks dependencies from memory; give it dated facts — try starlog.dev ↗ vet your agent's deps ↗ vibe-coding is fine. vibe-importing isn’t. — try starlog.dev ↗ vibe-importing isn’t fine ↗ your agent has never seen your private packages — try starlog.dev ↗ facts for private packages ↗ a linter for the dependencies your AI agent picks — try starlog.dev ↗ a linter for agent deps ↗ whois is redacted, cdns mask the rest — get the real operator — try whoisgeni.us ↗ who really runs that domain ↗ domain attribution that shows its work — full evidence chain — try whoisgeni.us ↗ domain intel w/ evidence ↗

← Back to Articles

Infinity: The Agent Runtime That Actually Hibernates

[ View on GitHub ]

Infinity: The Agent Runtime That Actually Hibernates

Hook

Most agent frameworks claim async concurrency while blocking the entire agent on every API call. Infinity actually deallocates agents between tool invocations, consuming literally zero memory while waiting—then reconstructs them from conversation history when results arrive.

Context

Agent frameworks have a dirty secret: they don't scale. LangChain, AutoGen, and most orchestration tools treat agents as persistent in-memory objects that await tool responses synchronously. Wrap a dozen HTTP calls in async/await syntax and you've created the illusion of concurrency, but each agent still occupies memory while idle, waiting for external APIs to respond. Run 1,000 concurrent agents and you're managing 1,000 Python objects with conversation state, tool contexts, and runtime overhead. Deploy to serverless and the problem becomes existential—Lambda functions time out while waiting for 30-second API calls, or you architect elaborate callback mechanisms that fracture your code across multiple functions.

The Hydro Project team—the distributed systems researchers behind the CALM theorem—recognized this as a fundamental architectural problem, not a library limitation. Infinity reimagines agents as hibernating entities that exist only as conversation history between invocations. When an agent calls a tool, it doesn't suspend or await—it serializes its conversation state, terminates completely, and consumes zero resources. When the tool completes (seconds, minutes, or hours later), the runtime reconstructs the agent from its conversation log and continues execution. This isn't clever memory management; it's a different execution model that treats conversation history as the source of truth for agent state.

Technical Insight

Resume Turn

Hibernation - Zero Resources

Active Turn

LLM request

serialize conversation

serialize state

tool invocation

terminate process

async execution

results ready

wake signal + results

restore conversation

restore state

resume agent

Agent Code

Infinity Runtime

Conversation Storage

State Storage

Reactive Agent Protocol

Agent Hibernated

Tool Executors

Runtime - New Process

Agent Continuation

System architecture — auto-generated

Infinity's architecture revolves around three components: conversation storage (the message history that defines an agent), state storage (arbitrary key-value data), and the Reactive Agent Protocol (RAP) for tool invocations. The magic happens in how these interact with Rust's async runtime. Traditional agent frameworks treat LLM calls and tool invocations as I/O operations within a continuous process. Infinity treats them as explicit hibernation boundaries.

Here's what agent code looks like:

use infinity::prelude::*;

#[infinity::agent]
async fn research_agent(ctx: AgentContext) -> Result<()> {
    let query = ctx.input::<String>()?;
    
    // Call LLM - agent hibernates here
    let plan = ctx.llm()
        .system("You plan research tasks")
        .user(&query)
        .generate()
        .await?;
    
    // Fire multiple tools concurrently
    let results = ctx.tools()
        .call("search_papers", json!({"query": plan}))
        .call("fetch_citations", json!({"query": plan}))
        .call("summarize_trends", json!({"query": plan}))
        .execute() // Agent hibernates, deallocates completely
        .await?; // Reconstructs when ALL tools complete
    
    // Process results
    let synthesis = ctx.llm()
        .user(&format!("Synthesize: {:?}", results))
        .generate()
        .await?;
    
    ctx.respond(synthesis)
}

The .await? points aren't suspending an async task—they're serialization boundaries. When execute().await runs, Infinity snapshots the conversation (system message, user query, LLM response with tool calls), writes it to storage, and terminates the agent's process. The three tool calls execute independently, potentially in different data centers or AWS regions. When all complete, the runtime spawns a new agent instance, replays the conversation to reconstruct context, and resumes at the await point with results injected as tool response messages.

This works because Infinity models agents as pure functions over conversation history. The AgentContext isn't holding conversation state in a struct—it's a view over an append-only message log. Every LLM call, tool invocation, and response is a message appended to this log. When reconstructing an agent, the runtime doesn't deserialize a complex object graph; it replays messages through the LLM context window. The agent's "memory" is literally the conversation history, making hibernation trivial: persist messages, drop everything else.

The Reactive Agent Protocol inverts traditional tool calling. Instead of agents pulling results via RPC:

// Traditional (blocks agent)
let result = http_client.get("api.example.com/tool").await?;

RAP makes tools push results through message delivery:

// RAP (agent can fully deallocate)
ctx.tools()
    .call("long_running_analysis", params)
    .execute() // Returns immediately, stores tool call message
    .await?; // Blocks hibernation, not execution

Under the hood, execute() publishes a message to the tool service with a callback address. The tool service processes asynchronously and delivers results to Infinity's ingress, which wakes the agent. This decoupling enables true serverless deployment—each agent "turn" is an independent Lambda invocation triggered by message arrival (initial input, tool completion, or LLM response).

The deployment abstraction is remarkably clean. Local development runs agents in a single process with in-memory message storage:

let runtime = infinity::Runtime::local()
    .with_storage(MemoryStorage::new())
    .build();

runtime.spawn_agent(research_agent).await?;

Production serverless deployment swaps storage and execution:

let runtime = infinity::Runtime::serverless()
    .with_storage(DynamoDbStorage::new())
    .with_executor(LambdaExecutor::new())
    .build();

// Same agent code, different infrastructure
runtime.spawn_agent(research_agent).await?;

The architectural isomorphism is intentional—hibernation boundaries are explicit in the programming model, so local and serverless behave identically. You can run 75,000 agents on a laptop because hibernated agents consume zero memory (just conversation records in storage). You can deploy the same code to Lambda because each agent turn is already designed as an independent invocation triggered by message arrival.

Infinity's MCP (Model Context Protocol) compatibility demonstrates strategic positioning. Existing MCP tool servers—designed for synchronous, single-process execution—gain hibernation and serverless deployment automatically. The runtime translates MCP tool calls into RAP messages, handles the hibernation lifecycle, and delivers results back through the MCP interface. This lets Infinity serve as infrastructure beneath existing agent ecosystems rather than requiring wholesale migration.

Gotcha

Conversation history grows without bounds. Every agent turn appends messages—LLM responses, tool calls, tool results—to an append-only log. A research agent making 50 tool calls across 10 turns creates 500+ messages. Eventually you hit context window limits (most models cap at 128k tokens) or storage costs balloon. Infinity provides no built-in summarization, pruning, or sliding window strategies. You're responsible for implementing conversation compaction, which requires domain-specific logic about what's safe to discard. This isn't a bug; it's inherent to conversation-as-state, but production systems need explicit memory management strategies that the framework doesn't provide.

Failure semantics are undefined for partial tool execution. If an agent fires 10 concurrent tool calls and 3 fail, what happens? The framework will reconstruct the agent with partial results, but there's no transactional rollback, compensation patterns, or declarative retry logic. You're writing imperative error handling in agent code that spans process boundaries. Distributed sagas, two-phase commit, or even simple "all-or-nothing" semantics require manual implementation. For systems where tool calls have side effects (sending emails, charging payments), this is a critical gap. RAP's push model also creates observability challenges—distributed tracing across hibernation boundaries requires external instrumentation because the agent's call stack is literally destroyed and reconstructed. Debugging "why did this tool call never complete?" means correlating message logs across services without automatic request IDs or parent spans.

Verdict

Use if: You're building multi-tenant agent systems where per-agent memory overhead kills scalability (SaaS with thousands of active users). You're deploying on serverless and need agents to survive across Lambda invocations without architectural gymnastics. Your agents spend most time blocked on slow external APIs (30+ second tool calls) where hibernation reclaims resources. You understand distributed systems trade-offs and can implement your own conversation pruning, failure compensation, and observability layers. Skip if: You're building single-agent desktop applications where one user talks to one agent in a single process—the hibernation overhead adds latency without benefit. You need real-time streaming responses where milliseconds matter (hibernation serialization adds 10-100ms). Your tools complete in under a second, making orchestration overhead exceed the savings. You want battle-tested production maturity with managed services, extensive integrations, and community support—Infinity is research-grade infrastructure from a brilliant team, but it's not LangChain's ecosystem or Microsoft's enterprise support. This is essential infrastructure for anyone treating agent concurrency seriously, not as async theater.