Fable5: Engineering AI Agent Reliability Through Enforcement, Not Hope

Hook

What if you stopped asking AI agents to 'remember' your methodology and started mechanically blocking non-compliant actions at the system call level?

Context

Anyone who's watched Claude or GPT-4 build software has seen the pattern: early in the conversation, it writes tests, considers edge cases, and follows your guidelines religiously. Three hours later, it's shipping untested code and claiming "everything works" when it demonstrably doesn't. The problem isn't capability—these models can reason about software quality. The issue is methodology decay under context pressure. As the conversation grows, your careful instructions get deprioritized against the immediate task demand.

The standard solution is better prompting: system messages, few-shot examples, periodic reminders. But this treats reliability as a training problem when it's actually an enforcement problem. If your CI pipeline relied on developers remembering to run tests rather than mechanically blocking merges without passing checks, you'd consider that architecture malpractice. Fable5-methodology applies the same thinking to AI agents: instead of hoping the model follows rules, it intercepts tool calls with shell hooks, spawns adversarial reviewer subagents with no edit permissions, and maintains append-only evidence logs that delivery gates read to catch false claims. It's a meta-programming framework that treats agent behavior like untrusted user input requiring validation at every boundary.

Technical Insight

The architecture implements a four-tier enforcement hierarchy that explicitly ranks mechanisms by reliability: lifecycle hooks (strongest), contracted subagents, lazy-loaded skills, and fallback context documents (weakest). Every rule in the methodology gets mapped to its strongest possible enforcer, refusing to leave critical behaviors as unenforceable prose.

At the foundation, shell scripts intercept MCP (Model Context Protocol) lifecycle events. The pre-tool-guard.py hook literally blocks dangerous operations before execution:

#!/bin/bash
# hooks/pre-tool-use.sh - runs before ANY tool execution

TOOL_NAME="$1"
ARGS_JSON="$2"

if [[ "$TOOL_NAME" == "bash" ]]; then
  # Extract command from args
  COMMAND=$(echo "$ARGS_JSON" | jq -r '.command')
  
  # Block destructive operations outside workspace
  if echo "$COMMAND" | grep -qE "rm -rf|sudo|mkfs"; then
    if [[ ! "$PWD" =~ /workspace/ ]]; then
      echo "BLOCKED: Destructive command outside workspace"
      exit 1
    fi
  fi
fi

# Log every tool call to append-only evidence log
echo "$(date -Iseconds)|$TOOL_NAME|$ARGS_JSON" >> evidence-log.txt
exit 0

This isn't advice that the model might forget—it's a mechanical gate that prevents execution. The hook layer solves the class of failures where agents do obviously wrong things (deleting system files, editing unchanged files, running tools with missing arguments).

The second tier uses contracted subagents for adversarial review. The qa-verifier agent contract explicitly removes write permissions, forcing independent verification:

# agents/qa-verifier.contract
name: qa-verifier
purpose: Independent test verification without implementation access

allowed_tools:
  - read_file
  - list_directory  
  - bash (read-only: git show, cat, grep)
  
denied_tools:
  - write_file
  - edit_file
  - create_file

contract:
  input_required:
    - test_file_path
    - claimed_results
  output_required:
    - independent_verification: bool
    - evidence: []
    - discrepancies: []
  must_refuse_if:
    - No test file exists
    - Claimed results not reproducible

This architectural choice—capability restriction rather than instruction—solves the observation versus assertion mismatch. The implementing agent can claim tests pass, but the QA verifier must independently reproduce results using only read tools. They can't collude because they literally can't both write.

The delivery gate ties it together by reading the evidence log to enforce proof of work:

#!/bin/bash
# hooks/delivery-gate.sh - runs when agent signals 'done'

EVIDENCE="evidence-log.txt"
STATUS_FILE=".delivery-status"

# Check: Were tests actually executed?
if ! grep -q "pytest\|npm test\|cargo test" "$EVIDENCE"; then
  echo "GATE FAILED: No test execution in evidence log"
  echo "claimed_done_without_testing" >> "$STATUS_FILE"
  exit 1
fi

# Check: Did independent QA verification run?
if ! grep -q "qa-verifier" "$EVIDENCE"; then
  echo "GATE FAILED: No independent verification"
  exit 1  
fi

# Check: Were files actually changed?
FILES_CLAIMED=$(cat task-plan.md | grep -c "edit:")
FILES_EDITED=$(grep -c "write_file\|edit_file" "$EVIDENCE")

if [[ $FILES_EDITED -lt $FILES_CLAIMED ]]; then
  echo "GATE FAILED: Claimed $FILES_CLAIMED edits, evidence shows $FILES_EDITED"
  exit 1
fi

echo "DELIVERY APPROVED"
exit 0

The innovation here is treating the append-only evidence log as ground truth. The agent can't claim "tests pass" without the log showing test execution. It can't say "implemented X" when the log shows no write operations. This solves the classic failure mode where models generate plausible status updates detached from actual actions taken.

The skills directory provides lazy-loaded procedural knowledge for complex patterns. The self-consistency-check.sh skill addresses a known failure mode where models satisfy individual constraints but miss their interactions:

# skills/self-consistency-check.sh
# Trigger: When requirements contain AND/OR logic or conflicting priorities

REQUIREMENTS=$(cat requirements.md)

# Extract all constraint statements
CONSTRAINTS=$(echo "$REQUIREMENTS" | grep -E "must|required|shall")

# For each pair, explicitly check interaction
echo "Running pairwise consistency check..."
echo "$CONSTRAINTS" | while IFS= read -r C1; do
  echo "$CONSTRAINTS" | while IFS= read -r C2; do
    if [[ "$C1" != "$C2" ]]; then
      echo "Can '$C1' and '$C2' both be satisfied? Prove with example."
      # This forces the model to consciously check rather than assume
    fi
  done
done

The real insight in the Difference Layer section is decomposing advanced model behavior into executable procedures. Instead of saying "be more careful," it defines nine cognitive patterns (predict-then-compare, read negative space, blast-radius-before-edits) as literal drills that weaker models can run deliberately. The hypothesis is that o1's apparent "thoughtfulness" partly comes from implicit execution of these patterns, and making them explicit through skills narrows the capability gap.

Gotcha

The enforcement architecture assumes infrastructure that most developers don't have. MCP lifecycle hooks (PreToolUse, PostToolUse, Stop) currently only work in specific execution environments—primarily Claude Desktop with MCP servers. You can't run this in ChatGPT, Claude web UI, or standard API integrations without re-engineering the entire interception layer. The shell hooks also assume POSIX environments, making Windows support non-trivial.

More fundamentally, the operational overhead is punishing. Every single edit triggers post-edit verification. Every 'done' signal hits the delivery gate. Non-trivial tasks require spawning multiple subagent sessions with full contract negotiation and independent verification. For a simple "add a function" task, you're looking at: main agent proposes implementation, code-reviewer runs cold-read analysis, qa-verifier independently reproduces tests, delivery-gate checks evidence log. That's 4x the token consumption and latency of just letting the model work. The methodology is valuable when correctness absolutely matters and you're burning tokens anyway; it's absurd for rapid prototyping or exploratory coding.

The "self-enforcing" claim also oversells what's possible. Hooks can block rm -rf outside workspace, but they can't detect semantic failures—implementing the wrong requirement correctly, gold-plating features, or subtle logic bugs that pass tests. The subagent independence is somewhat illusory too: if code-reviewer and qa-verifier run on the same base model, they share training distribution blind spots. True adversarial review needs capability diversity (mix GPT-4 and Claude), not just process isolation.

Verdict

Use if: You're building multi-agent orchestration systems and need design patterns for enforcing behavior through capability restriction rather than instruction; you're researching AI reliability through process enforcement and want a worked example of treating methodology as infrastructure with explicit SLOs; you're working in regulated domains where you need mechanical proof that agents followed procedures; or you're trying to make GPT-4-class models behave more like o1 through procedural scaffolding and explicit cognitive drills. Skip if: You need production-ready tooling for standard development workflows (the MCP dependency and operational overhead make this a research artifact, not a product); you're using the methodology with weaker models where even perfect process enforcement won't bridge fundamental capability gaps; you're building consumer-facing agents where sub-second response times matter; or you work in environments without shell access and MCP server infrastructure. The real value here is the conceptual framework—evidence logs that block unsupported claims, contracted subagents with adversarial constraints, and the discipline of mapping every rule to its strongest enforcer. Steal those patterns for your own scaffolding rather than running this system verbatim.