> your AI agent picks dependencies from memory; give it dated facts — try starlog.dev ↗ vet your agent's deps ↗ vibe-coding is fine. vibe-importing isn’t. — try starlog.dev ↗ vibe-importing isn’t fine ↗ your agent has never seen your private packages — try starlog.dev ↗ facts for private packages ↗ a linter for the dependencies your AI agent picks — try starlog.dev ↗ a linter for agent deps ↗ whois is redacted, cdns mask the rest — get the real operator — try whoisgeni.us ↗ who really runs that domain ↗ domain attribution that shows its work — full evidence chain — try whoisgeni.us ↗ domain intel w/ evidence ↗

← Back to Articles

RealityDiff: Git Semantics for Tracking What You Believed and When You Changed Your Mind

[ View on GitHub ]

RealityDiff: Git Semantics for Tracking What You Believed and When You Changed Your Mind

Hook

Your news feed updates constantly, but your brain doesn't commit those changes to history. When a breaking story reverses course, can you trace exactly what made you 90% confident on Tuesday morning—and why you're only 30% sure by Friday?

Context

Journalists, researchers, and analysts face a unique knowledge management problem: they need to track claims that evolve as new evidence emerges. Traditional note-taking tools treat each edit as destructive—you update a document and lose the previous state. Git solves this for code, but running git diff on prose produces textual diffs that miss semantic changes. When you're tracking "Did Apple acquire an AI startup?" you don't care that someone rephrased a sentence; you care that new SEC filings dropped confidence from 75% to 40%.

Existing solutions fall into two camps. Research tools like Roam or Obsidian give you bidirectional links and daily notes, but version control requires manual git discipline and diffs show line-by-line changes, not confidence deltas. Collaborative forecasting platforms like Metaculus provide probability tracking with crowd wisdom, but you can't run private investigations or feed in custom evidence streams. RealityDiff bridges this gap by treating claims as versioned objects with git-inspired workflows—commits only happen when beliefs meaningfully change, and you can blame a confidence spike or revert a rumor-driven update.

Technical Insight

The architecture appears deceptively simple until you recognize it's not using git internals at all—it's a custom append-only log that borrows git's command vocabulary. Each claim lives as a JSON document in ~/.local/share/realitydiff/ with a structure like:

{
  "id": "apple-ai-acquisition",
  "claim": "Apple acquired an AI startup in Q4 2024",
  "confidence": 0.62,
  "evidence": [
    {
      "source": "TechCrunch article",
      "url": "https://...",
      "excerpt": "Sources familiar with...",
      "weight": 0.3,
      "timestamp": "2024-01-15T09:23:00Z"
    }
  ],
  "commits": [
    {
      "sha": "a3f2b1c...",
      "timestamp": "2024-01-15T09:30:00Z",
      "confidence_delta": 0.15,
      "reasoning": "New TechCrunch report citing insider sources"
    }
  ]
}

When the background watcher polls for updates (via xAI's Grok API with real-time X search), it doesn't blindly append evidence. The system computes a semantic diff by asking the LLM: "Given this new evidence, how does confidence change?" Only when the delta crosses a threshold—say, ±0.10 confidence or a new evidence category (legal filings vs. social media rumors)—does it write a commit. This is the killer feature: you get version history without noise from cosmetic rewrites.

The CLI mimics git commands with epistemology-aware semantics:

# Initialize tracking for a new claim
realitydiff init "Apple acquired an AI startup"

# Show confidence evolution over time
realitydiff log apple-ai-acquisition
# Output:
# commit a3f2b1c (2024-01-15 09:30)
# Confidence: 0.47 → 0.62 (+0.15)
# New evidence: TechCrunch insider report

# Diff two time periods
realitydiff diff --since="2 weeks ago" apple-ai-acquisition
# Shows confidence trajectory, new evidence categories, contradictory sources

# Blame a confidence spike
realitydiff blame --confidence=0.85 apple-ai-acquisition
# Traces which evidence pushed you to 85% certainty

# Revert a rumor-driven panic
realitydiff revert a3f2b1c apple-ai-acquisition

Background monitoring runs via OS-native schedulers instead of Python daemons. On macOS, it generates a launchd plist that invokes the CLI's watch command every N minutes. On Linux, it creates a systemd timer. This design choice is brilliant for personal research tools—zero memory footprint when idle, survives reboots without custom init scripts, and uninstalls cleanly without orphaned processes.

The web UI is a lightweight Python HTTP server (likely FastAPI based on ecosystem trends) serving static HTML/JS. When you visit http://localhost:8080/claims/apple-ai-acquisition, the frontend polls REST endpoints that query the SQLite backend. The clever bit: you can browse claims "as of a commit" by filtering evidence arrays by timestamp, effectively giving you git checkout semantics for your belief state. Want to see what you knew before the panic-inducing tweet? Just rewind to the previous commit hash.

The xAI API integration does heavy lifting for evidence collection. When you initialize a claim, you provide search terms and the watcher periodically queries Grok with: "Find recent web articles and X posts about [claim]. Extract evidence supporting or refuting it." The LLM returns structured JSON with source URLs, excerpts, and suggested confidence adjustments. This passive intelligence collection means you can track 10 unfolding stories without manually refreshing Google News—the system becomes your research assistant that commits only when something meaningful changes.

Gotcha

The xAI API dependency is an architectural single point of failure with no escape hatch. If Grok's search quality degrades, your evidence stream degrades. If xAI changes pricing or rate limits, your background monitoring breaks. There's no fallback to Perplexity, Exa, or traditional search APIs, and no offline mode once you've initialized claims. This is acceptable for personal research tools where you control the infrastructure, but it makes RealityDiff unsuitable for teams or long-term archival projects where API longevity matters.

The linear commit history model breaks down for competing hypotheses. If you're tracking "Apple's AI strategy," you might have evidence for both "acquiring a startup" and "building in-house." RealityDiff forces you to create separate claims for each hypothesis—there's no branching like git, no way to merge evidence streams later when the truth becomes clear. This works fine for binary claims ("Did X happen?") but struggles with multi-factor investigations where reality often lands somewhere between competing narratives.

Confidence scoring is LLM-generated, not mathematically rigorous. There's no Bayesian updating, no calibration curves, no accounting for base rates or contradictory evidence weights. When Grok says confidence jumped from 0.60 to 0.75, that number reflects the model's vibes, not a defensible probability. For casual research this is fine—you want directional signals, not forecasting tournament precision. But if you're doing due diligence where "62% confident" needs to mean "62 out of 100 similar situations would confirm," you need proper prediction infrastructure like Metaculus or custom Bayesian tooling.

Verdict

Use if: You're a solo journalist, investigative researcher, or analyst tracking 3-10 unfolving stories where you need defensible audit trails of belief evolution. The git-inspired CLI maps perfectly to "why did I believe this last Tuesday?" questions, and automated evidence collection via Grok saves hours of manual RSS trawling. The single-command installer and OS-native scheduling make it frictionless for personal use. Skip if: You need team collaboration (the SQLite backend in ~/.local/share breaks multi-user scenarios with no merge resolution), real-time alerts (poll-based watching means minutes of staleness), or rigorous probability math (LLM confidence scores aren't calibrated). Also skip if xAI API costs or lock-in concern you—there's no provider abstraction, so you're betting on Grok's longevity and search quality. For teams, use Notion with database views; for calibrated forecasting, use Metaculus; for full control, build custom Obsidian workflows with Zapier.