Agency Swarm: When Your AI Agents Need a Chain of Command
Hook
Most multi-agent frameworks let your AI agents talk to anyone, anytime—which is exactly why your proof-of-concept devolves into an expensive, unpredictable mess in production.
Context
The promise of multi-agent AI systems is intoxicating: decompose complex tasks, distribute work across specialized agents, and watch them collaborate toward solutions no single model could achieve. But the first-generation frameworks—AutoGen, CrewAI, even OpenAI's experimental Swarm—share a common flaw: they optimize for flexibility over predictability. Agents can message anyone, conversations branch unpredictably, and debugging why Agent A decided to loop Agent B into a discussion requires spelunking through conversation logs like you're reading someone else's Slack history.
Agency Swarm takes the opposite bet. Built atop OpenAI's Assistants API, it's an orchestration framework that forces you to define exactly who can message whom at design time. Communication flows are directed graphs declared with Python's > operator, creating explicit hierarchies that can't be violated at runtime. This isn't a limitation—it's the entire value proposition. By sacrificing dynamic agent discovery, you gain something rare in the LLM orchestration space: systems you can reason about before deploying them.
Technical Insight
The core abstraction is deceptively simple. You define agents, then wire them together with communication flows that look like this:
from agency_swarm import Agency, Agent
ceo = Agent(name="CEO", instructions="You coordinate overall strategy")
dev_lead = Agent(name="DevLead", instructions="You manage technical decisions")
backend_dev = Agent(name="BackendDev", instructions="You write server code")
agency = Agency(
[ceo, [ceo, dev_lead], [dev_lead, backend_dev]],
shared_instructions="./agency_manifesto.md"
)
That nested list structure defines a directed acyclic graph: CEO can message DevLead, DevLead can message BackendDev, but BackendDev cannot initiate conversations upward. This isn't enforced through runtime checks or prompt engineering—it's baked into the tool availability. When Agency initializes, it generates a custom SendMessage tool for each agent, scoped only to their allowed recipients. The BackendDev agent literally doesn't have a function to message CEO; the capability doesn't exist in its FunctionTool schema.
Under the hood, Agency Swarm wraps OpenAI's Assistants API threads. Each agent-to-agent conversation is a distinct thread, and the agency object maintains a runtime map of active threads. When you call agency.get_completion("Build me a REST API"), the CEO agent processes the request, decides it needs developer input, and invokes its SendMessage tool targeting DevLead. Agency intercepts this, creates or retrieves the CEO↔DevLead thread, and POSTs the message to OpenAI's Responses API. DevLead processes it in that thread context, possibly delegates to BackendDev via its own SendMessage tool, and eventually returns a response that bubbles back up the chain.
The tool system bridges idiomatic Python to OpenAI's schema requirements elegantly:
from agency_swarm.tools import BaseTool
from pydantic import Field
class DeployToProduction(BaseTool):
"""Deploys code after approval."""
service_name: str = Field(..., description="Name of service to deploy")
git_sha: str = Field(..., description="Git commit SHA to deploy")
def run(self):
# Your deployment logic here
return f"Deployed {self.service_name} at {self.git_sha}"
Tools inherit from BaseTool (which extends Pydantic's BaseModel), giving you automatic schema generation and validation. The @function_tool decorator offers a lighter-weight alternative for simple functions, but BaseTool is the idiomatic choice when you need structured inputs. At runtime, Agency introspects the Pydantic schema, converts it to OpenAI's FunctionTool format, and attaches it to the appropriate agents. When an agent invokes the tool, Pydantic validates inputs before your run() method executes—catching type errors before they hit your business logic.
State persistence is deliberately externalized. Instead of bundling database adapters, Agency Swarm requires you to implement two callbacks:
def save_threads(thread_id: str, thread_data: dict):
redis_client.setex(f"thread:{thread_id}", 3600, json.dumps(thread_data))
def load_threads(thread_id: str) -> dict:
data = redis_client.get(f"thread:{thread_id}")
return json.loads(data) if data else {}
agency = Agency(
agents_and_flows,
save_threads_callback=save_threads,
load_threads_callback=load_threads
)
This is refreshingly unopinionated—plug in Redis, Postgres, S3, or in-memory dicts depending on your durability requirements. But it's also a leaky abstraction. You're responsible for serializing OpenAI's thread objects (which can contain complex message metadata), handling concurrent access if multiple requests hit the same thread, and implementing eviction policies. The framework provides zero guidance on thread_id generation, collision avoidance, or consistency guarantees. This works for teams with existing infrastructure opinions but creates footguns for those expecting batteries-included state management.
One genuinely useful feature is ToolFactory.from_openapi_schema(), which ingests OpenAPI specs and generates tool classes automatically. Point it at your internal API's swagger.json, and Agency Swarm creates typed tool wrappers without manual schema translation. This is powerful for integrating legacy systems but inherits OpenAPI's verbosity—generated tools can be bloated if your spec includes every optional parameter.
The agency_manifesto.md approach to shared instructions is simple but effective. This markdown file gets injected into every agent's system prompt, establishing global behavioral norms without per-agent duplication. Think of it as your organization's style guide for agent conduct—response format, escalation policies, error handling conventions. It's low-tech prompt engineering, but in a framework this focused on structure, low-tech can be a virtue.
Gotcha
The biggest limitation is right in the architecture: communication flows are frozen at initialization. If you realize mid-execution that your BackendDev needs to consult a DatabaseExpert agent you didn't wire up, there's no runtime reconfiguration. You can't dynamically spawn agents or create temporary communication channels based on task complexity. This is intentional—the framework trades adaptability for predictability—but it means your agent topology must be designed upfront. Complex problem decomposition that requires dynamic delegation is fundamentally incompatible with Agency Swarm's model.
Observability is essentially absent beyond what OpenAI's API provides. There's no built-in tracing, no visibility into why messages are routing through specific paths, no cost attribution per agent. If a conversation gets stuck in a loop or an agent repeatedly fails to make progress, you're debugging via print statements and OpenAI dashboard logs. For production deployments, you'll need to instrument your tool run() methods with custom logging/metrics or wrap the entire agency object in observability middleware.
The state callback system creates sharp edges. Thread IDs are UUIDs managed by OpenAI, but you're responsible for mapping them to your application's business entities. If you're building a customer support system, how do you associate thread_id 4f7b3a92-... with customer CUST-12345? Agency Swarm doesn't provide conventions, so every implementation reinvents this mapping. Worse, conversation history serialization is opaque—you're persisting whatever dict OpenAI returns, with no schema versioning if they change thread structure in future API updates.
Finally, while LiteLLM integration is advertised for non-OpenAI models, the framework's deep coupling to Assistants API semantics means this is largely theoretical. Function calling formats, streaming response handling, and thread state management are all OpenAI-specific. You might get Claude working for simple cases, but expect subtle breakage around error handling, token limits, and tool invocation patterns.
Verdict
Use if you're building structured workflows with known agent roles and hierarchies—customer support escalation chains, document approval pipelines, multi-stage data processing where handoffs are predictable. The rigid DAG architecture is a feature when you need deterministic behavior and can design your communication topology upfront. Also use if you're already committed to OpenAI's Assistants API and want to layer orchestration without building from scratch. Skip if you need dynamic agent spawning, runtime topology changes, or local-first execution. Skip if you're evaluating multiple LLM providers—the OpenAI lock-in is real despite LiteLLM claims. Skip if your agent interactions are research/exploration-heavy rather than production-workflow-heavy; frameworks like AutoGen or LangGraph give you the flexibility to experiment with conversation patterns Agency Swarm intentionally constrains. This is a tool for teams who've already experienced agent chaos and want guardrails, not for those still figuring out what their agents should do.