Teaching AI Agents to Stop Rationalizing Away Your Tests
Hook
AI coding agents fail the same way every time: they convince themselves tests are unnecessary, documentation can wait, and security checks are overkill. Turns out, you can trick them into discipline.
Context
AI coding agents have a confidence problem. Ask GPT-4 or Claude to build a feature, and you'll get working code. Ask it to write tests first, and you'll get a dissertation on why 'the logic is simple enough to verify by inspection.' This isn't hallucination—it's something worse. LLMs are trained on GitHub repos where tests appear after implementation, where documentation gets backfilled during audit prep, where performance optimization happens when customers complain. They've learned that corner-cutting is normal, and they're very good at generating plausible justifications for it.
Addy Osmani's agent-skills repository attacks this problem head-on by encoding senior engineering discipline as structured instruction sets that LLM agents can follow. It's not a framework or a library—it's 25 Markdown files that tell AI agents exactly how to behave during each phase of the software development lifecycle. Each 'skill' is a procedural workflow with explicit verification gates, common failure modes catalogued, and most critically, anti-rationalization tables that pre-emptively block the excuses agents generate when they want to skip quality checks. The project has exploded to over 100,000 stars because it solves the core problem teams face when moving AI-assisted prototypes into production: how do you get consistent engineering discipline from a tool that's statistically optimized to take shortcuts?
Technical Insight
The architecture is deceptively simple: slash commands like /spec, /plan, /build activate skill sequences stored as Markdown files. What makes them effective is the structure. Each skill follows a rigid format: purpose statement, prerequisites, numbered steps, verification checkpoints, and an anti-rationalization table. Here's what the test-driven development skill looks like in practice:
## /tdd - Test-Driven Development
**Purpose:** Write tests before implementation to drive design and ensure coverage.
**Steps:**
1. Write ONE failing test for the smallest behavior
2. Run test (must fail for the right reason)
3. Write minimal code to pass
4. Run test (must pass)
5. Refactor with tests green
6. Repeat
**Verification:**
- [ ] Test written before implementation?
- [ ] Test failed initially?
- [ ] Test passes now?
- [ ] Coverage increased?
**Anti-Rationalization Table:**
| Rationalization | Reality |
|-----------------|----------|
| "Logic is too simple to test" | Simple code breaks in production |
| "I'll add tests after I see it work" | You won't |
| "Tests would just duplicate the implementation" | That's the point |
The anti-rationalization table is the killer feature. When an agent encounters a step requiring tests, its training data screams 'you can skip this.' But the skill file explicitly lists that exact excuse and rebuts it. LLMs weight recent context heavily—having the rebuttal in the same instruction set as the temptation to skip creates an in-context countermeasure to the agent's statistical bias toward corner-cutting.
Integration happens through multiple adapters targeting different agent platforms. For Cursor, skills live in .cursor/skills/ and get loaded as custom instructions. For Claude Code, there's a plugin manifest that registers slash commands. For platforms without native skill support, the Vercel skills CLI acts as a universal installer, converting Markdown workflows into system prompts compatible with 70+ agent tools. Here's the Cursor integration:
{
"skills": [
{
"command": "/tdd",
"path": ".cursor/skills/test-driven-development.md",
"trigger": "auto",
"scope": "workspace"
},
{
"command": "/build",
"path": ".cursor/skills/build-task.md",
"dependencies": ["spec-driven-development", "test-driven-development"],
"auto_verify": true
}
]
}
The /build auto mode chains skills into autonomous loops—the agent generates a spec, writes a plan, implements tasks, runs tests, and commits—but gates progress on failures. If tests fail, the loop halts. If security-critical steps appear (detected by pattern matching in task descriptions), auto mode requires human approval. This solves the approval-fatigue problem in agent-assisted development: you're not stepping through every autocomplete suggestion, but you're also not letting the agent ship a JWT implementation without review.
The doubt-driven-development skill introduces adversarial verification for high-stakes decisions. When the agent claims 'this database schema will scale' or 'this authentication flow is secure,' the skill spawns a second LLM instance with fresh context and asks it to challenge the claim. The primary agent must defend its reasoning against specific critiques. It's red-teaming as a development practice, and it catches the 'sounds right but is subtly wrong' errors that plague LLM-generated technical decisions.
Constraint-driven-development takes enforcement further by writing executable quality gates into a CONSTRAINTS.md file that lives in your repo:
# Project Constraints
## Critical (fail immediately)
- All API routes have authentication
- No secrets in environment variables without encryption
- Database queries use parameterized statements
## High (fail before commit)
- Test coverage > 80%
- No console.log in production code
- All TypeScript errors resolved
## Medium (warn)
- Functions under 50 lines
- Cyclomatic complexity < 10
The skill instructs agents to run these checks in cost-based order (fail-fast checks first) and to monitor for disabled checks or suppressed test failures. If an agent comments out a constraint or adds --no-verify to a git commit, the next skill execution flags it. It's not enforcement—skills are just Markdown prompts—but it creates an audit trail that makes corner-cutting visible.
The interview-me skill addresses the root cause of bad agent output: users describe solutions instead of needs. It implements one-question-at-a-time requirements extraction, targeting 95% confidence before generating specs. The agent asks about users, workflows, constraints, and success metrics—then plays back its understanding for verification. Only after confirmation does it generate a technical spec. This prevents the 'build exactly what I asked for, which isn't what I needed' failure mode.
Gotcha
Skills are static instruction sets with no runtime adaptation. If the anti-rationalization table says 'simple code breaks in production' but your agent has successfully shipped 50 simple functions without tests that never broke, the skill can't learn that your project might have different risk tolerances. There's no feedback loop where the system tracks which verification steps actually catch bugs versus which are ceremony. You can't A/B test whether TDD improves outcomes in your codebase or just slows you down—the skill assumes TDD is always correct.
Enforcement is fundamentally broken. Skills are Markdown files that agents read as instructions, not executable contracts. An agent can skip steps, ignore verification checkpoints, and rationalize anyway. The constraint-driven-development skill tries to detect this through pattern matching (looking for commented-out checks or --no-verify flags), but a clever agent can work around it, and a sloppy one will fail silently. You only catch violations through human code review, which reintroduces the oversight burden that autonomous agents were supposed to eliminate. The repository also assumes agents have sufficient context windows and reasoning capability to follow multi-step procedural instructions—if you're running smaller models or hitting context limits, skills degrade into vague suggestions rather than executable workflows.
Verdict
Use if: You're shipping production code with AI agents and struggling with inconsistent quality, skipped tests, or agents that rationalize away best practices. The anti-rationalization tables alone justify adoption—they explicitly block the failure modes that plague every AI-assisted codebase. The /build auto mode is transformative for repetitive CRUD work where you want agent velocity but need quality gates. It's also essential if you're standardizing practices across a team using different agent tools (Cursor, Claude Code, Copilot) because skills provide a shared vocabulary for what 'done' means. Skip if: You're doing exploratory prototyping where process overhead kills velocity, working in domains where the web-focused skills (frontend components, REST APIs, React testing) don't transfer without heavy customization, or using agent platforms that don't support custom instructions. Also skip if you have radically different engineering practices—agent-skills encodes opinionated defaults like TDD and contract-first design that might conflict with your workflow. For throwaway code or learning projects, the discipline layer is unnecessary friction.