Building Agent Memory with Git: How Filesystem-as-Database Powers Persistent AI Context

Hook

What if your AI agent's memory was just a git repository? No vector databases, no embeddings—just markdown files that accumulate knowledge like a second brain.

Context

AI agents have a memory problem. Every time you start a new chat session with Claude, ChatGPT, or any LLM-powered assistant, you're back to square one. The agent doesn't remember your projects, your collaborators, your coding style, or the experiments you abandoned last month. You end up copy-pasting context into every conversation, maintaining the same background information across dozens of interactions.

Vector databases and semantic search tools have emerged to solve this—Pinecone, Weaviate, and purpose-built agent memory stores like Mem0 promise to give agents long-term recall. But these solutions require infrastructure: embedding models, similarity searches, database maintenance. The personal-monorepo-template takes a radically simpler approach: what if agent memory was just a git repository with a thoughtful directory structure? Projects are markdown files in projects/. People are markdown files in people/. Skills are Python scripts in .codex/skills/. The filesystem becomes the database, git becomes the audit log, and your text editor becomes the admin interface.

Technical Insight

The architecture centers on convention over configuration. When Anthropic's Codex agent (an autonomous AI assistant that can execute code and call APIs) starts a task, it reads from this monorepo to understand context. The directory structure is deliberately flat and semantic:

personal-monorepo/
├── .codex/
│   └── skills/           # Python modules Codex can execute
├── projects/             # Active work, long-lived context
├── experiments/          # Ephemeral prototypes
├── people/               # Collaborators, contacts
└── threads/              # Conversation histories

The magic happens in .codex/skills/. These aren't configuration files—they're executable Python modules that extend what Codex can do. Here's the pattern from the onboarding skill:

# .codex/skills/onboard_services.py
import anthropic_client
import os

def execute():
    """Introspect connected services to bootstrap vault"""
    
    # Pull GitHub repos and create project files
    repos = anthropic_client.github.list_repos()
    for repo in repos:
        if repo.activity_last_90_days > 5:
            write_project_file(repo)
    
    # Extract frequent collaborators from Slack
    slack_users = anthropic_client.slack.get_frequent_contacts()
    for user in slack_users:
        write_people_file(user)
    
    # Mine sent emails for writing style
    sent_messages = anthropic_client.gmail.get_sent(limit=100)
    extract_writing_patterns(sent_messages)

def write_project_file(repo):
    path = f"projects/{repo.name}.md"
    with open(path, 'w') as f:
        f.write(f"# {repo.name}\n\n")
        f.write(f"**Status:** {repo.status}\n")
        f.write(f"**Last Active:** {repo.last_commit}\n\n")
        f.write(repo.description)

This solves the cold-start problem brilliantly. Instead of manually documenting every project and contact, the onboarding skill introspects services you already use—GitHub, Slack, Gmail—and generates initial markdown files. The agent builds its own context by reading the digital exhaust of your actual work.

The separation between projects/ and experiments/ is subtle but crucial. Projects are durable—the web app you've been maintaining for two years, the side business with recurring customers. Experiments are ephemeral—that weekend hack to try a new framework, the prototype that never shipped. By keeping them separate, you prevent dead experiments from polluting the context window when Codex is reasoning about active work. It mirrors how real engineers think: "Is this a project or just an experiment?"

The write-like-me-bootstrap skill is where things get linguistically interesting:

# .codex/skills/write_like_me_bootstrap.py
def extract_writing_patterns(messages):
    """Generate stylistic prompts from sent message corpus"""
    
    # Filter to substantial messages (>100 words)
    substantial = [m for m in messages if len(m.body.split()) > 100]
    
    # Extract patterns
    patterns = {
        'avg_sentence_length': calculate_avg_sentence_length(substantial),
        'common_openers': extract_common_openers(substantial),
        'sign_offs': extract_sign_offs(substantial),
        'technical_density': measure_jargon_ratio(substantial)
    }
    
    # Generate few-shot examples
    exemplars = select_representative_messages(substantial, patterns)
    
    # Write to .codex/prompts/writing-style.md
    write_style_prompt(patterns, exemplars)

This is corpus linguistics applied to agent outputs. By analyzing your sent messages, the system creates a stylistic fingerprint—sentence length, technical density, common phrases. When Codex drafts an email or documentation on your behalf, it consults writing-style.md to match your voice. It's few-shot learning materialized as a file.

Thread automations implement recurring agent behaviors through scheduled skill invocations. A thread in threads/daily-standup.md might have automation metadata:

# Daily Standup

**Automation:** Run `.codex/skills/summarize_progress.py` at 9:00 AM
**Context:** All files in `projects/` with status:active

## 2024-01-15
[Agent-generated summary appears here]

Codex becomes a cron job that reasons. At 9 AM, it executes the skill, reads relevant project files, generates a summary, and appends it to the thread—all visible in git history. You're not just getting agent outputs; you're getting versioned, auditable agent memory.

The filesystem-as-database pattern has surprising advantages. You can grep for context: grep -r "API rate limit" projects/ finds every project that's hit rate limiting issues. You can use standard diff tools to see how the agent's understanding evolved: git diff HEAD~10 people/jane-doe.md shows what Codex learned about Jane over the last 10 commits. You can manually edit any file—fix a mistake, add context the agent couldn't infer, archive old experiments. There's no API, no query language, no ORM. Just files.

Gotcha

The entire architecture is locked to Anthropic's Codex platform, which isn't publicly available at the time of writing. Without Codex, you have a well-organized collection of markdown files but none of the agent automation—no skill execution, no service introspection, no automated memory updates. It's like having a car without an engine. If you're using AutoGPT, LangChain agents, or any other agent framework, you'd need to rewrite the skill execution layer entirely.

The filesystem-as-database pattern also has scaling limits that will hit you around 500-1000 markdown files. Codex needs to scan directories to find relevant context, and there's no indexing beyond the operating system's file cache. If you have 200 projects and 300 people files, "find everyone I've worked with on machine learning projects" requires reading hundreds of files and doing string matching in memory. Vector databases handle this elegantly with semantic search—you'd query by meaning, not grep patterns. The template has no story for archiving, no notion of 'cold' vs. 'hot' memory, no automatic summarization of old context. Memory accumulates linearly forever.

Security is also underspecified. Skills are arbitrary Python executed by Codex with filesystem access and API credentials. There's no sandboxing, no permission model, no code review workflow. If a skill is compromised or has a bug, it could corrupt your entire memory vault or leak credentials to external services. The template assumes you trust Codex completely and write bulletproof skills—a risky assumption for an agent that's supposed to operate autonomously.

Verdict

Use if: You have access to Anthropic's Codex platform and work across multiple long-lived projects where you need the agent to remember context across sessions without manual re-prompting. The onboarding automation is legitimately valuable—it mines GitHub, Slack, and email to bootstrap context instead of forcing manual documentation. The git-based versioning gives you an audit trail of how agent memory evolves, and the ability to manually edit markdown files keeps you in control. If you're comfortable with the filesystem-as-database tradeoff and won't exceed a few hundred context files, this gives you durable agent memory with zero infrastructure. Skip if: You don't have Codex access (this is completely platform-locked), need semantic search over thousands of memory items (use Mem0 or Zep instead), want portable agent memory that works with multiple LLM platforms (try LangChain memory modules), or require production-grade security for skills that touch sensitive APIs. The architectural elegance is real, but it only matters if you're in Anthropic's ecosystem.