Teaching AI to Browse: A Workshop Implementation of Frozen Agent Training

Hook

What if the only way your AI agent could improve at browser automation was by reading a markdown file — and it had no memory of its previous attempts?

Context

Modern AI agents typically learn through fine-tuning weights, accumulating memory in vector stores, or building up conversational context. But there's a compelling alternative that's often overlooked: teaching agents through pure prompt engineering, where capability improvement happens entirely outside the execution layer. This approach — sometimes called 'frozen agent training' — treats the executor as a stateless function that gets better solely because you've refined the instructions it receives.

The ai-engineer-autobrowse repository implements this pattern for browser automation tasks. Built as workshop infrastructure for teaching meta-learning concepts, it orchestrates a two-agent system: a 'teacher' that observes failures and edits strategy documents, and a 'doer' that executes browser automation without ever knowing about its own learning process. The doer is architecturally prevented from improving itself — it's frozen, vendored from Browserbase's skills toolkit, and runs identically on every iteration. All capability gains manifest through iterative edits to a strategy.md file that gets injected into the doer's system prompt. This constraint transforms a complex reinforcement learning problem into observable gradient descent over text documents.

Technical Insight

The system architecture reveals three distinct layers that communicate through filesystem artifacts rather than shared memory. At the foundation sits the Browserbase browse CLI — a wrapper around Playwright-over-CDP sessions running in cloud-hosted browsers. This handles the gnarly infrastructure problems: anti-bot evasion, session management, network isolation. The frozen doer agent consumes this CLI and executes tasks based purely on its system prompt plus the current strategy document.

Above that runs the teacher loop, powered by Claude Opus, which implements single-hypothesis hill climbing. Here's the core iteration logic:

while (!taskSucceeded && iterations < MAX_ATTEMPTS) {
  // Execute task with current strategy
  const trace = await executeBrowserTask({
    task: userGoal,
    strategy: readStrategyFile(),
    sessionId: browserbaseSession
  });
  
  // Teacher analyzes failure
  const hypothesis = await claude.messages.create({
    model: 'claude-opus-4',
    messages: [{
      role: 'user',
      content: `Previous attempts:\n${trace}\n\nStrategy:\n${currentStrategy}\n\nGenerate ONE specific improvement.`
    }]
  });
  
  // Apply single edit to strategy
  const updatedStrategy = await applyHypothesis(currentStrategy, hypothesis);
  writeStrategyFile(updatedStrategy);
  
  // Judge if we're done
  taskSucceeded = await evaluateSuccess(trace);
  iterations++;
}

The deliberate constraint here is "ONE specific improvement" — the teacher can't hedge bets with multiple theories or explore branches in parallel. This greedy search is pedagogically valuable because it makes the learning gradient visible. You can literally diff strategy.md across iterations and watch capabilities emerge: first the agent learns to wait for page loads, then it discovers CSS selectors are more reliable than text matching, then it realizes some sites need authentication state.

The third layer is a Node.js server streaming SSE updates to a browser UI that embeds the live Browserbase session alongside dual-pane agent reasoning. Every intermediate artifact — traces, hypotheses, strategy diffs — gets serialized to disk under workspace// directories in real-time. This isn't just logging; it's the primary interface for understanding what the meta-learning system is actually doing.

When a task finally succeeds, the system graduates the strategy into a reusable SKILL.md file exported to .claude/skills/. These files follow a specific format designed for copy-paste into Claude Code or Codex tool-use systems:

# SKILL: Login to Dashboard

## Trigger Conditions
- User asks to access authenticated areas
- URL matches pattern: */login or */signin

## Strategy
1. Wait for page.networkidle before interacting
2. Use CSS selectors over text matching (ids > classes > tags)
3. Handle both username+password and email+password forms
4. Verify success by checking for redirect or disappearance of login form

## Known Edge Cases
- CAPTCHA pages: Return failure immediately, don't retry
- 2FA prompts: Document but don't automate

The architecture makes a pragmatic split: hard infrastructure (browser sessions, anti-detection) stays in Browserbase's cloud, while the teaching logic runs locally with filesystem state. This means you can iterate on meta-learning prompts or evaluation criteria without wrestling with browser automation internals. The frozen agent constraint also creates natural interfaces — the doer exposes a pure function from (task, strategy) to trace, making it trivial to replay scenarios or test counterfactual strategies.

One subtle design choice: the system couples tight rather than batching. Each hypothesis immediately triggers a new execution attempt instead of accumulating multiple theories to test in parallel. This optimizes for workshop observability — humans can follow the reasoning — at the cost of computational efficiency. You're paying for full Opus inference on every iteration, which pencils out for teaching but would be ruinous at scale.

Gotcha

The single-hypothesis hill climbing guarantee suboptimal convergence. Once the teacher commits to a strategy direction that happens to work for simple cases, there's no backtracking mechanism when that approach hits a ceiling on harder variants. For example, if early iterations succeed using text matching on buttons, the strategy might never discover that some sites require waiting for JavaScript frameworks to hydrate. You'd need to manually reset and provide different initial conditions to escape local maxima.

The cost model is brutal for experimentation. At $10-15 per training run with Opus, iterating on teaching prompts or testing edge cases gets expensive fast. There's no warm-start mechanism to resume from checkpoints, no way to replay cached traces, and no fallback to cheaper models for obvious improvements. The frozen agent constraint also means strategies can't encode task-specific state — every execution starts from scratch with only static instructions. This works for atomic tasks like "log in" or "search for product" but breaks down for complex workflows that need conditional branching or dynamic adaptation mid-execution.

Verdict

Use if: You're an AI engineer exploring outer-loop learning patterns and want a concrete reference implementation of frozen agent training; you're running workshops on meta-learning concepts and need observable infrastructure where students can watch strategy gradients emerge; you need to extract reusable browser automation primitives in the SKILL.md format for Claude-based toolchains; you're researching how much capability can be encoded in prompt engineering alone without weight updates or memory. Skip if: You need reliable production browser automation (just use Playwright or Puppeteer directly with GPT-4V); you want to train skills at scale (the cost model doesn't work beyond a handful of tasks); you require generalization beyond the exact trained scenario; you're looking for end-user automation tooling rather than AI engineering infrastructure. This is workshop-grade code that demonstrates a valuable pattern you'd reimplement with better search algorithms and cheaper models, not something you'd deploy as-is.