> your AI agent picks dependencies from memory; give it dated facts — try starlog.dev ↗ vet your agent's deps ↗ vibe-coding is fine. vibe-importing isn’t. — try starlog.dev ↗ vibe-importing isn’t fine ↗ your agent has never seen your private packages — try starlog.dev ↗ facts for private packages ↗ a linter for the dependencies your AI agent picks — try starlog.dev ↗ a linter for agent deps ↗ whois is redacted, cdns mask the rest — get the real operator — try whoisgeni.us ↗ who really runs that domain ↗ domain attribution that shows its work — full evidence chain — try whoisgeni.us ↗ domain intel w/ evidence ↗

← Back to Articles

OpenExecutive: Building an AI C-Suite with Cached Prompts and Episodic Memory

[ View on GitHub ]

OpenExecutive: Building an AI C-Suite with Cached Prompts and Episodic Memory

Hook

The fastest way to get fired by an LLM is to let it auto-send emails as your CEO. OpenExecutive's 'Act as me' feature solves this by drafting Gmail replies in your voice but forcing manual approval—a small UX detail that separates toy demos from production tools.

Context

Most AI assistant tools are glorified chatbots with RAG bolted on. You upload some PDFs, ask questions, and get answers that forget everything by next Tuesday. This works fine for 'What's our PTO policy?' but falls apart for executive work: strategic decisions span weeks, require specialist knowledge (legal, finance, HR), and compound on past choices. A founder scaling from 10 to 50 people needs something between a $200/month ChatGPT subscription and a $200k chief of staff—an always-on assistant that remembers decisions, routes complex queries to specialist reasoning, and integrates with the actual tools execs live in (Gmail, Slack, Google Drive).

OpenExecutive fills this gap with a hierarchical multi-agent system where a Claude Sonnet orchestrator routes queries to nine specialist agents (CSO, CFO, CHRO, CTO, etc.), each backed by domain-specific knowledge and Opus-grade reasoning for high-stakes queries. The clever bit isn't just agent routing—it's the dual-layer RAG that separates curated MBA-level frameworks (git-tracked markdown) from user-uploaded company docs, episodic memory that extracts decisions into SQLite for cross-session recall, and integrations that turn the assistant into a participant in your workflow (Slack threads, email drafts, scheduled reminders). It's FastAPI + Next.js, designed for self-hosting by teams who want control over their data and don't trust SaaS tools with board meeting notes.

Technical Insight

FastAPI Lifespan

Parallel Tool Calls

User Query

Claude Sonnet Orchestrator

Dual-Layer RAG

MBA Docs builtin/

Company Docs ChromaDB

Cached System Prompt

85% hit rate

9 Specialist Agents

CSO CFO CHRO...

Response Synthesis

Haiku Episodic Memory

SQLite

Decisions + Tasks

Integration Layer

Slack Socket

Discord Gateway

Gmail/Outlook Pollers

Drive/OneDrive/Confluence

Next.js 15 + Auth.js

System architecture — auto-generated

The architecture is a masterclass in working within LLM constraints rather than fighting them. OpenExecutive's prompt caching strategy hits 85% cache rates by treating the system prompt as immutable layers: persona definition, company profile, and knowledge index are separate cached blocks that never change mid-session. RAG context gets injected into the user turn, not the system prompt, so dynamic document retrieval doesn't invalidate the cache.

Here's how the orchestrator routes a query like 'Should we switch to unlimited PTO?' to the CHRO agent:

# Simplified from agents/orchestrator.py
async def route_query(user_message: str, context: CompanyContext):
    tools = [agent.as_tool() for agent in SPECIALIST_AGENTS]
    
    response = await anthropic.messages.create(
        model="claude-sonnet-4-0",
        system=[
            {"type": "text", "text": PERSONA_PROMPT, "cache_control": {"type": "ephemeral"}},
            {"type": "text", "text": context.company_profile, "cache_control": {"type": "ephemeral"}},
            {"type": "text", "text": context.knowledge_index, "cache_control": {"type": "ephemeral"}}
        ],
        messages=[
            {"role": "user", "content": f"{retrieve_docs(user_message)}\n\n{user_message}"}
        ],
        tools=tools,
        parallel_tool_use=True
    )
    
    # Execute tool calls in parallel, synthesize into single voice
    if response.stop_reason == "tool_use":
        specialist_responses = await asyncio.gather(*[
            execute_agent(call.name, call.input) for call in response.content if call.type == "tool_use"
        ])
        return synthesize_executive_voice(specialist_responses)

The CHRO agent gets invoked as a tool call, retrieves from both builtin/hr/pto_frameworks.md (git-tracked) and synced Google Docs about current policies, then returns analysis. The orchestrator synthesizes all specialist responses into a single coherent answer that sounds like one executive, not a committee.

Episodic memory is a pragmatic trade-off: instead of embedding every message (expensive, noisy), a background Haiku pass extracts structured decisions after each response:

# Post-response memory extraction
async def extract_decision(conversation_id: str, response: str):
    extraction = await anthropic.messages.create(
        model="claude-haiku-4-0",
        messages=[{
            "role": "user",
            "content": f"Extract any decisions, commitments, or action items from this executive response:\n\n{response}"
        }]
    )
    
    if decision := parse_decision(extraction.content):
        db.execute(
            "INSERT INTO decisions (conversation_id, decision_text, timestamp) VALUES (?, ?, ?)",
            (conversation_id, decision, datetime.now())
        )

Next session, past decisions get replayed in a <past_decisions> XML block in the user turn (again, not the system prompt, to preserve cache). This means the CFO agent 'remembers' that you approved a $50k marketing budget last month when evaluating a new spend request.

The scheduler enforces exactly-once semantics using SQLite's UPDATE...RETURNING, a pattern borrowed from Postgres job queues:

# Single-instance job claiming without race conditions
def claim_next_job(worker_id: str):
    return db.execute("""
        UPDATE scheduled_tasks 
        SET status = 'claimed', worker_id = ?, claimed_at = CURRENT_TIMESTAMP
        WHERE id = (
            SELECT id FROM scheduled_tasks 
            WHERE status = 'pending' AND scheduled_for <= CURRENT_TIMESTAMP
            ORDER BY scheduled_for ASC LIMIT 1
        )
        RETURNING *
    """, (worker_id,)).fetchone()

This works beautifully for single-instance deployments but becomes a scalability dead-end—horizontal scaling requires rearchitecting around Redis or accepting duplicate executions.

The integration layer is where product thinking shines. Instead of microservices, everything runs as FastAPI lifespan coroutines sharing ChromaDB and SQLite:

@asynccontextmanager
async def lifespan(app: FastAPI):
    tasks = [
        slack_socket_listener(),
        discord_gateway(),
        gmail_poller(),
        drive_sync_daemon(),
        scheduler_worker()
    ]
    async with asyncio.TaskGroup() as tg:
        for task in tasks:
            tg.create_task(task)
        yield

The 'Act as me' Gmail drafting is defensively designed—it uses the owner's OAuth token to create drafts, never auto-sends:

async def draft_reply(thread_id: str, draft_content: str, owner_token: str):
    service = build('gmail', 'v1', credentials=owner_token)
    message = create_mime_message(draft_content)
    draft = service.users().drafts().create(
        userId='me',
        body={'message': {'raw': base64.urlsafe_b64encode(message.as_bytes()).decode(), 'threadId': thread_id}}
    ).execute()
    return draft['id']  # User reviews and sends manually

This threading of the needle between automation and control is what makes the tool usable for high-stakes executive communication.

Gotcha

The single-instance scheduler is the most obvious scalability wall. SQLite's UPDATE...RETURNING works for one API process but breaks the moment you add a second instance—you'll get duplicate job execution or need to bolt on Redis/Postgres locking, which defeats the simplicity of SQLite. There's no migration path in the codebase, so you're stuck rewriting the scheduler if you outgrow one server.

Prompt caching assumes stable documents, but that assumption fails under real-world document churn. Sync a 500-doc Confluence space and the knowledge index block invalidates on every ingestion batch, thrashing cache hit rate from 85% to near-zero. The cost savings evaporate until syncing stabilizes, and there's no incremental index update—it's full rebuild every time. Episodic memory has the opposite problem: it's append-only SQLite with no semantic search, so you can't ask 'what did we decide about pricing last quarter' without full-text search or manually scrolling through decision logs. The system replays decisions sequentially in the prompt, which works for dozens of decisions but will blow context windows at hundreds.

The fat API process runs 14+ background coroutines with zero observability—a crash in the Drive sync daemon silently breaks document ingestion until you notice stale data. There's no health checks, no per-task metrics, no automatic restarts. Production deployments need external monitoring or you'll debug invisible failures.

Verdict

Use if: You're a founder or small exec team (5-50 people) drowning in Slack questions and need an always-on assistant that remembers past decisions, routes complex queries to specialist reasoning, and integrates with Google Workspace/Slack without uploading everything to a SaaS vendor. The episodic memory and dual-layer RAG justify self-hosting effort if you make high-stakes decisions that compound over months. Skip if: You need horizontal scaling (the scheduler is a single-instance bottleneck), workflow orchestration beyond simple reminders (no state machines or saga patterns for multi-step approvals), or fine-grained access control (the people table is flat, no org hierarchy). If you have >10k docs in Confluence/Drive, expect cache thrashing and relevance tuning. For repetitive tasks without strategic reasoning, Lindy.ai's no-code workflows will ship faster. For production-grade reliability, Dust.tt's Temporal-based orchestration scales better but lacks the product polish.