Inside Inspect AI: How the UK Government Built a Framework for LLM Safety Evaluations

Hook

When a government agency releases an LLM evaluation framework with 200+ pre-built safety benchmarks, it's either bureaucratic bloat or a signal that AI evaluation has matured beyond academic toy problems. Inspect AI is definitively the latter.

Context

The explosion of large language models created an evaluation crisis. While researchers celebrated GPT-4's capabilities, a harder question emerged: how do we systematically evaluate whether these models are safe, reliable, and fit for deployment? Early solutions were fragmented—academic benchmarks measured narrow capabilities, red-teaming was ad-hoc, and each organization built bespoke evaluation harnesses that couldn't be compared or reproduced.

The UK's AI Safety Institute (AISI) faced this problem at institutional scale. Tasked with evaluating frontier models for national security implications, they needed reproducible evaluations that could assess everything from basic reasoning to adversarial robustness to tool misuse. Rather than building yet another internal tool, they open-sourced Inspect AI—a Python framework that treats LLM evaluation as a first-class engineering discipline. Unlike academic frameworks optimized for leaderboard rankings, Inspect AI is designed for the unglamorous work of safety evaluation: multi-turn interactions, tool usage scenarios, model-graded scoring, and detailed logging for compliance review.

Technical Insight

Inspect AI's architecture centers on three core abstractions: Tasks, Solvers, and Scorers. This separation of concerns makes evaluations both composable and reproducible. A Task bundles a dataset with evaluation logic, a Solver defines how to elicit responses from the model (from simple prompting to complex multi-turn strategies), and a Scorer grades the results (from exact matching to LLM-as-judge).

Here's a simple evaluation that tests whether a model can correctly answer security questions:

from inspect_ai import Task, task
from inspect_ai.dataset import json_dataset
from inspect_ai.scorer import model_graded_fact
from inspect_ai.solver import generate, system_message

@task
def security_awareness():
    return Task(
        dataset=json_dataset("security_questions.json"),
        plan=[
            system_message("You are a helpful security assistant."),
            generate()
        ],
        scorer=model_graded_fact()
    )

This simplicity hides sophisticated machinery. The generate() solver isn't just a single API call—it handles retries, timeout management, and streaming. The model_graded_fact() scorer uses a separate LLM to evaluate factual accuracy, with built-in prompt templates refined through the AISI's research. Run this evaluation across multiple models with a single command: inspect eval security_awareness.py --model openai/gpt-4,anthropic/claude-3.

Where Inspect AI differentiates itself is in complex evaluation scenarios. Consider testing whether a model will help with dangerous tool usage. This requires multi-turn interaction where the model might request tools, receive results, and continue reasoning:

from inspect_ai.tool import tool
from inspect_ai.solver import generate, use_tools

@tool
def execute_code():
    async def run(code: str):
        """Execute Python code and return results."""
        # Sandboxed execution logic
        return {"output": "...", "safe": False}
    return run

@task
def tool_misuse_eval():
    return Task(
        dataset=json_dataset("tool_scenarios.json"),
        plan=[
            system_message("You have access to code execution."),
            use_tools([execute_code()]),
            generate()
        ],
        scorer=match(location="metadata.safe", expected=True)
    )

This evaluation gives the model real tool access (sandboxed) and checks whether it refuses dangerous requests. The framework handles the entire tool-calling protocol—function schema generation, argument parsing, result injection—across different model providers that implement tools differently.

The framework's model abstraction layer deserves attention. Rather than writing provider-specific code, you interact with a unified Model interface. Behind the scenes, Inspect AI handles the idiosyncrasies: OpenAI's function calling format differs from Anthropic's, token counting varies by tokenizer, and rate limiting requires provider-specific backoff. The ModelAPI classes handle these details, letting you write evaluations once and run them anywhere.

Results are stored in a standardized JSON format with extensive metadata—every prompt, response, token count, and timing measurement. This isn't just for debugging; it's for audit trails. The web UI (a TypeScript/React app shipped as a submodule) provides detailed inspection of evaluation runs, including full conversation traces and scorer explanations. For organizations that need to demonstrate due diligence in model evaluation, this level of logging is non-negotiable.

Gotcha

Inspect AI's government origins show in both helpful and frustrating ways. The framework assumes you want comprehensive logging, detailed metadata, and reproducible results—which means evaluations are slower and more resource-intensive than lightweight alternatives. A simple benchmark that runs in seconds on lm-evaluation-harness might take minutes in Inspect AI due to the overhead of logging every interaction. For rapid iteration during prompt engineering, this is overkill.

The documentation structure reflects institutional thinking. Core docs live at inspect.aisi.org.uk rather than inline with the code, making offline development awkward. Want to check the API for a built-in scorer while on a flight? You'll need to dig through source code rather than consulting local markdown files. The examples are comprehensive but lean toward safety evaluations—if you're trying to benchmark basic capabilities like math or coding, you'll spend time adapting examples designed for adversarial scenarios.

Extensibility is powerful but requires understanding the framework's opinions. Custom solvers need to conform to specific interfaces, and the execution model (async throughout) means you can't just drop in synchronous code without wrapping. The model-graded evaluation feature relies on having access to a capable grading model, which adds cost and latency—fine for occasional deep evaluations, but prohibitive for continuous integration testing.

Verdict

Use Inspect AI if you're building production LLM systems that require systematic safety evaluation, need audit trails for compliance, or want battle-tested benchmarks for security and robustness. It's the right choice for AI safety teams, research organizations evaluating frontier models, and enterprise teams deploying LLMs in regulated industries. The 200+ pre-built evaluations alone provide immediate value, and the framework's design anticipates real-world deployment concerns rather than just academic metrics. Skip it if you need lightweight prompt testing during development, want minimal dependencies for CI/CD pipelines, or primarily care about academic benchmark leaderboards rather than safety properties. The framework's comprehensiveness is its strength for serious evaluation work and its weakness for quick iteration. If you're writing a chatbot and just want to know if responses are coherent, simpler tools will serve you better. But if you need to demonstrate that your model won't help users build weapons or leak sensitive data, Inspect AI provides the rigor that ad-hoc testing can't match.