Why Your AI Coding Agent Keeps Forgetting What It Built: The Five-Subsystem Harness Solution

Hook

Your AI agent writes perfect code for 20 minutes, then starts breaking its own tests. The problem isn't the model—it's that you're treating a stateful software engineering system like a stateless chatbot.

Context

AI coding assistants like Claude, Cursor, and GitHub Copilot have moved from autocomplete toys to agents that can scaffold entire features. But anyone running these tools on real codebases hits the same wall: the agent starts strong, then drifts. It rewrites code it just finished. It skips verification steps. It 'forgets' architectural decisions from three prompts ago. The naive solution is prompt engineering—add more instructions, be more specific, include more examples. This is treating the symptom, not the disease.

Harness engineering emerged from Anthropic and OpenAI's research into why agents fail on software engineering tasks. The insight: coding isn't a single-turn task. It's a stateful, multi-step workflow with verification gates, context that spans sessions, and scope constraints that prevent feature creep. The solution isn't better prompts—it's better infrastructure. A harness is the scaffolding around an AI agent that maintains state, enforces verification, and constrains scope. The walkinglabs/learn-harness-engineering repository packages this research into a hands-on curriculum, teaching the five-subsystem harness pattern through six progressive projects built on the same Electron codebase.

Technical Insight

The repository's core teaching is that harnesses aren't prompt templates—they're a five-subsystem architecture. Each subsystem addresses a specific agent failure mode that prompt engineering can't fix.

Instructions (AGENTS.md, CLAUDE.md): These aren't feature requests. They're operational contracts defining how the agent should work. A typical AGENTS.md includes architectural constraints ("Always write TypeScript with strict type checking"), verification requirements ("Run npm test before marking work complete"), and scope boundaries ("One feature per session, logged in feature_list.json"). The course teaches you to version these files alongside code, making them evolve with your project rather than staying static.

State (progress.md, feature_list.json, git history): This is where most DIY harnesses fail. Developers rely on git commits alone, but agents can't parse git history effectively. The course teaches explicit state artifacts. Here's a progress.md pattern from Project 3:

# Session 2024-01-15: Tag Management Feature

## Completed
- Added tag schema to database (commit a1b2c3d)
- Implemented tag CRUD operations in API layer
- All tests passing (npm test at 14:32)

## In Progress
- Frontend tag selector component (40% complete)
- Blocked: Need to decide on multi-select vs autocomplete UI

## Next Session
- Resolve UI decision (review Figma mocks)
- Complete tag selector component
- Add tag filtering to search

This isn't for humans—it's a structured context artifact the agent reads at session start. The course shows how this prevents the "rewrite working code" failure mode: the agent sees what's complete and verified, not just what exists in files.

Verification (test/lint/type-check gates): The harness enforces a checklist before any work is considered done. Project 4 introduces init.sh and cleanup.sh scripts that automate verification:

#!/bin/bash
# cleanup.sh - Run before committing or ending session

echo "Running verification pipeline..."

# Type checking
npm run type-check || { echo "Type check failed"; exit 1; }

# Linting
npm run lint || { echo "Lint failed"; exit 1; }

# Tests
npm test || { echo "Tests failed"; exit 1; }

# Build validation
npm run build || { echo "Build failed"; exit 1; }

echo "All checks passed. Safe to commit."
git status

The agent gets instructions to run this script before marking work complete. This isn't novel—it's standard CI/CD. The insight is making it an explicit harness subsystem rather than assuming the agent will 'know' to verify its work.

Scope (single-feature work constraints): The course's most controversial pattern is aggressive scope limiting. Each session targets one feature, logged in feature_list.json with status tracking. If the agent starts expanding scope mid-session, the harness template includes explicit intervention instructions: "Stop. Log this new idea in feature_list.json as a future task. Return to the current feature." This trades flexibility for reliability—you get predictable incremental progress instead of ambitious half-finished features.

Session Lifecycle (init/cleanup checklists): Every session starts with init.sh (pull latest, install deps, read progress.md) and ends with cleanup.sh (run verification, update progress.md, commit if clean). The course teaches you to make these checklists explicit in your AGENTS.md so the agent follows them without prompting.

The pedagogical strategy is unconventional: each project's solution becomes the next project's starter code. Project 2's starter is literally Project 1's completed state. This snapshot-based progression makes harness evolution concrete. You see the same Electron app gain progressively more sophisticated harness infrastructure across six iterations, making it obvious how each subsystem prevents specific failure modes that manifested in earlier projects.

Gotcha

This repository is educational content, not tooling. You're not npm installing a package—you're copy-pasting markdown templates and adapting them to your codebase. The course provides no CLI tool to generate harness files, no programmatic API to customize templates for different languages or frameworks. You get the pattern; implementation is entirely manual. For teams wanting ready-to-use frameworks, this is frustrating.

The Electron app teaching vehicle creates setup friction that undermines accessibility. You need Node, Electron, and a desktop environment to run the projects. The course justifies this by saying Electron has 'real-world product complexity,' but simpler CLI tools or web apps would demonstrate the same harness patterns with less environmental overhead. If you're working in Python, Go, or Rust, you'll spend cognitive energy translating TypeScript/Electron examples rather than focusing on harness concepts. The repository lacks quantitative validation. Despite citing Anthropic's research (which includes controlled experiments showing harnesses improve task success rates), this course provides zero benchmarks. You don't know if these five subsystems actually outperform simpler approaches, or if all five are necessary versus just state + verification. It's prescriptive without being empirical—'do it this way' without proving this way is optimal.

Verdict

Use if: You're already running AI coding agents (Claude, Cursor, GitHub Copilot) on production codebases and hitting reliability problems—agents that drift mid-session, skip tests, or lose context across multiple work sessions. This course directly addresses those failure modes with concrete file structures and workflow patterns you can implement today. It's best for teams willing to invest setup time (6 projects × 2-4 hours each) to learn systematic harness design rather than wanting quick fixes. Skip if: You're looking for a ready-to-use framework or library to npm install (this is educational templates, not runtime tooling), you work in languages other than TypeScript/JavaScript and don't want translation overhead (all examples are Electron/Node), you want empirical comparisons of different harness designs (it's prescriptive, not evaluative), or you're still optimizing prompts rather than building infrastructure (this explicitly rejects prompt engineering as the solution). For teams wanting actual frameworks instead of patterns, evaluate SWE-agent or OpenHands—they provide runtime harnesses with benchmarks, not educational content.