Teaching AI to Build AI: Google's agents-cli and the Meta-Tooling Revolution

Hook

What if the future of developer tooling isn't building better tools, but teaching AI coding assistants how to use the tools we already have?

Context

The agent development landscape has become a paradox. On one hand, frameworks like LangChain, LlamaIndex, and Google's own Agent Development Kit (ADK) have made it technically possible to build production agents. On the other, the cognitive overhead of learning these frameworks—understanding their abstractions, orchestration patterns, deployment requirements, and cloud integrations—remains brutally high. The typical path involves reading documentation, experimenting with examples, fighting dependency conflicts, and eventually deploying something fragile to production.

Google's agents-cli takes a fundamentally different approach. Instead of simplifying the ADK or abstracting away its complexity, it treats AI coding assistants (GitHub Copilot, Cursor, Cline) as first-class execution environments. The tool consists of two components: a CLI for scaffolding, evaluation, and deployment, and a 'skills' system that injects structured knowledge directly into your coding assistant's context window. The result is meta-tooling—software that teaches software how to write software. It's purpose-built for the Gemini Enterprise Agent Platform and makes no apologies for being tightly coupled to Google Cloud. This isn't a generic agent framework; it's a deployment pipeline disguised as a productivity tool.

Technical Insight

The architecture of agents-cli reveals a fascinating inversion of control. Traditional tools execute commands; agents-cli educates the thing that generates the commands. The 'skills' system is the key innovation here. When you run npx @googlecloud/agents-cli skills add, you're not installing executable code—you're downloading structured markdown documentation that your AI coding assistant reads and interprets.

Here's what a typical workflow looks like. You start a conversation with your coding assistant and say 'create a new agent project.' Because the skills are indexed in your assistant's context, it knows to generate code that runs:

# Your AI assistant generates this command
agents-cli init my-retail-agent --template basic
cd my-retail-agent

This scaffolds a project with opinionated structure: agent.py for your agent logic, eval/ directory for test scenarios, and configuration files for Google Cloud deployment. The scaffolding isn't revolutionary, but what happens next is. Your coding assistant now has access to skills documentation that explains ADK patterns, Cloud Run deployment configurations, and Vertex AI integration—all formatted as LLM-readable reference material.

When you tell your assistant 'add a tool that queries BigQuery,' it doesn't just generate generic Python. It generates ADK-compliant code using the correct decorators and patterns:

from google.adk.tools import tool
from google.cloud import bigquery

@tool
def query_sales_data(
    query: str,
    project_id: str = "my-project"
) -> str:
    """Query BigQuery for sales analytics.
    
    Args:
        query: SQL query to execute
        project_id: GCP project containing the dataset
    
    Returns:
        Query results as formatted string
    """
    client = bigquery.Client(project=project_id)
    result = client.query(query)
    return str([dict(row) for row in result])

The coding assistant knows to use the @tool decorator, proper type hints for ADK's schema generation, and docstrings that become part of the agent's function-calling interface—because the skills documentation taught it these patterns.

The evaluation subsystem is where things get architecturally interesting. Most agent frameworks treat evaluation as an afterthought—maybe some unit tests, perhaps a benchmark dataset. agents-cli implements LLM-as-judge at infrastructure scale:

# Generate synthetic test scenarios
agents-cli eval dataset synthesize \
  --scenario "customer returns policy questions" \
  --num-examples 50 \
  --output eval/returns_scenarios.json

# Run evaluation with adaptive rubrics
agents-cli eval run \
  --dataset eval/returns_scenarios.json \
  --rubric eval/rubrics/accuracy.yaml \
  --judge gemini-1.5-pro

# Cluster failures to identify patterns
agents-cli eval analyze \
  --results eval/results/latest.json \
  --cluster-failures

This generates a failure cluster report showing that 23% of failures involve date parsing in return windows, 18% fail on policy exceptions for electronics, and so on. It's agentic testing for agents—using LLMs to generate test cases, using LLMs to grade outputs, using embeddings to cluster failure modes.

The eval optimize command attempts automated prompt engineering by analyzing which prompt variations correlate with better eval scores. This is either brilliant or naive depending on your eval rubric quality, but it's telling that Google is building prompt optimization directly into the deployment pipeline.

Deployment is where the Google Cloud lock-in becomes explicit:

# Deploy to Cloud Run with one command
agents-cli deploy \
  --project my-gcp-project \
  --region us-central1 \
  --enable-tracing

Under the hood, this is orchestrating Cloud Run deployment, setting up Cloud Trace integration, configuring IAM permissions for Vertex AI access, and wiring up the Gemini Enterprise Agent Platform APIs. The simplicity is seductive, but you're trading command brevity for infrastructure understanding. There's no Terraform state, no Kubernetes manifests, no visibility into what's actually being provisioned.

The 'enhance' and 'upgrade' commands are particularly telling:

# Add capabilities to existing project
agents-cli enhance --feature streaming-responses

# Upgrade to newer ADK version
agents-cli upgrade --target-version 0.8.0

These suggest Google is managing schema evolution across a rapidly changing agent stack. The ADK is pre-GA, APIs are in flux, and agents-cli is attempting to paper over breaking changes with automated migration. This is a real operational problem—agent frameworks at this maturity level tend to break backward compatibility frequently—but it also means you're trusting opaque upgrade logic with your production code.

Gotcha

The skills system's elegance is also its fundamental weakness: behavior is non-deterministic and model-dependent. Claude 3.5 Sonnet interprets skills documentation differently than GPT-4, which means the same natural language prompt generates different ADK code depending on which coding assistant you're using. There's no testing story for skills themselves—you can't validate that a skill will consistently guide assistants to generate correct code. You discover skill limitations by having your coding assistant generate broken agents.

The evaluation infrastructure has a transparency problem. LLM-as-judge is only as good as the rubrics, and 'adaptive rubrics' is hand-wavy without seeing the actual rubric adjustment logic. The eval system can tell you that your agent scored 7.3/10 on 'accuracy' according to Gemini 1.5 Pro's judgment, but there's no ground truth validation. If the judge model has biases or misunderstands your domain, those biases propagate into your optimization loop. The failure clustering is useful for pattern recognition, but it doesn't tell you whether the patterns represent actual production issues or artifacts of synthetic test generation.

Deployment is a one-way door with no obvious exit. The agents-cli deploy command doesn't generate reusable infrastructure-as-code—it's an imperative operation that mutates cloud state. There's no evidence of blue-green deployments, canary releases, or traffic splitting. Rolling back appears to mean re-deploying an older version, which is fine for experimentation but terrifying for production systems. Multi-cloud or self-hosted deployment isn't just unsupported; it's architecturally impossible. This tool is a Google Cloud sales funnel where the local development experience (using AI Studio API) is a gateway to cloud deployment lock-in.

Verdict

Use agents-cli if you're already committed to Google Cloud and Gemini Enterprise, you're comfortable with AI coding assistants as your primary development interface, and you value velocity over infrastructure control. The skills system is genuinely novel for organizations where 'teaching the AI' is more scalable than training developers. The evaluation clustering is more sophisticated than most agent benchmarking, and if you need to move fast on Google's agent platform, this is the fastest path. Skip it entirely if you need multi-cloud portability, production-grade deployment controls (canary releases, rollbacks, traffic splitting), or transparent infrastructure-as-code. Skip it if you're skeptical of LLM-as-judge evaluation or need deterministic build processes. And definitely skip it if Google's pre-GA disclaimer concerns you—this is beta tooling for a beta platform, and you'll be debugging both simultaneously. For teams who want the same velocity without GCP lock-in, writing raw ADK code with proper Terraform modules gives you agents-cli's benefits without its constraints.