AIVSS: The First CVSS-Style Framework for Scoring AI System Vulnerabilities

Hook

A prompt injection vulnerability in a customer service chatbot and a buffer overflow in Apache get the same CVSS score of 0.0 because traditional frameworks can't quantify AI-specific risks. AIVSS changes that.

Context

Security teams have relied on the Common Vulnerability Scoring System (CVSS) for nearly two decades to quantify software vulnerabilities, enabling consistent risk assessment across organizations. But CVSS was designed for traditional software where vulnerabilities are deterministic code flaws—SQL injection, buffer overflows, authentication bypasses. When you deploy an LLM that hallucinates customer data, exhibits bias in loan decisions, or leaks training data through membership inference attacks, CVSS offers no guidance. There's no metric for "how badly can this model be poisoned" or "what's the blast radius of a prompt injection attack."

This gap has created chaos in AI security. Organizations are deploying language models in production with no standardized way to compare the severity of a fine-tuning data poisoning attack versus a jailbreak vulnerability versus catastrophic model drift. Security teams accustomed to triaging vulnerabilities by CVSS scores are suddenly flying blind when asked whether the adversarial robustness of their image classifier represents a critical or medium risk. AIVSS (Artificial Intelligence Vulnerability Scoring System) attempts to bridge this chasm by extending CVSS methodology into AI-specific territory, creating quantifiable metrics for the unique attack surfaces that neural networks introduce.

Technical Insight

AIVSS builds on CVSS's familiar three-metric structure—Base, Environmental, and Temporal—but radically expands what those metrics measure. The Base Score still evaluates attack vectors, complexity, and required privileges, but adds nine AI-specific weighted factors: model robustness (0.0-1.0), data sensitivity, ethical implications, decision criticality, adaptability, adversarial attack surface, AI lifecycle vulnerability, AI governance, and cloud security alignment with CSA's LLM Threat Taxonomy.

The scoring formula combines these dimensions multiplicatively, which means vulnerabilities compound rather than average out. A simplified version looks like:

# AIVSS Base Score Calculation (simplified)
base_score = (
    attack_vector * attack_complexity * privileges_required
)

ai_specific_weight = (
    model_robustness * 0.15 +
    data_sensitivity * 0.15 +
    ethical_implications * 0.10 +
    decision_criticality * 0.15 +
    adaptability * 0.10 +
    adversarial_surface * 0.15 +
    lifecycle_vulnerability * 0.10 +
    governance * 0.05 +
    cloud_security * 0.05
)

environmental_score = (
    confidentiality_requirement * 
    integrity_requirement * 
    availability_requirement
)

model_complexity_multiplier = 1.0  # ranges 1.0-1.5 based on model size

final_score = (
    base_score * 
    ai_specific_weight * 
    environmental_score * 
    model_complexity_multiplier
)

Each metric uses detailed rubrics. Model robustness, for example, scores 0.9-1.0 if "the model demonstrates minimal resilience to adversarial examples or distribution shifts," dropping to 0.0-0.3 for "robust defenses including adversarial training, input validation, and certified defenses." This inverted scale means higher numbers indicate greater vulnerability, matching CVSS conventions.

The data sensitivity metric specifically evaluates what kind of information the model was trained on and could leak. A model trained on publicly available Wikipedia data scores low (0.0-0.3), while one trained on PII, financial records, or trade secrets scores high (0.7-1.0). This directly addresses membership inference and model inversion attacks that have no analog in traditional software.

Decision criticality acknowledges that not all AI failures carry equal weight. A recommendation engine suggesting the wrong movie is annoying; an autonomous vehicle misclassifying a pedestrian is catastrophic. The framework scores decisions that "directly impact human safety, financial stability, or critical infrastructure" at 0.9-1.0, while "non-critical applications with minimal user impact" score 0.0-0.3.

The lifecycle vulnerability metric is particularly novel, scoring risks across the entire ML pipeline:

lifecycle_stages = {
    'data_collection': {
        'poisoning_risk': 0.8,  # can attackers inject bad training data?
        'bias_introduction': 0.6,
        'privacy_leakage': 0.7
    },
    'training': {
        'backdoor_injection': 0.9,
        'model_stealing': 0.5,
        'hyperparameter_manipulation': 0.4
    },
    'deployment': {
        'adversarial_examples': 0.8,
        'prompt_injection': 0.9,
        'api_abuse': 0.7
    },
    'monitoring': {
        'drift_detection_failure': 0.6,
        'feedback_loop_poisoning': 0.7
    }
}

lifecycle_score = max([max(stage.values()) for stage in lifecycle_stages.values()])

The framework's integration with the Cloud Security Alliance's LLM Threat Taxonomy makes it immediately relevant for modern deployments. It explicitly maps to threats like "LLM01: Prompt Injection," "LLM02: Insecure Output Handling," and "LLM03: Training Data Poisoning," giving security teams a bridge between high-level threat categories and quantified risk scores they can prioritize in sprints.

What makes AIVSS particularly valuable is its model complexity multiplier (1.0-1.5). A simple logistic regression model might get a 1.0 multiplier, while a 175-billion parameter transformer gets 1.4-1.5. This acknowledges an uncomfortable truth: larger, more capable models have exponentially larger attack surfaces, more opaque decision-making, and higher governance overhead. Your GPT-4-class model inherently presents more risk than a decision tree, even if you've applied identical security controls.

Gotcha

The elephant in the room: AIVSS is a framework on paper, not a tool you can run. The repository contains detailed scoring rubrics and methodology documentation, but no scanner, no GitHub Action, no Python package that actually assesses your models. You can't point it at your MLflow registry and get vulnerability scores. Every assessment requires manual evaluation by someone who understands both AI security and your specific system architecture. For a security team already stretched thin, this is a significant barrier to adoption.

The multiplicative scoring formula also creates interpretation challenges. When you combine nine weighted factors, each ranging 0.0-1.0, with base scores, environmental metrics, and complexity multipliers, you can end up with final scores that are difficult to contextualize. Is a 0.73 high or medium? How does that compare to a CVSS 7.3? The framework doesn't provide severity categorization bands (like CVSS's Low/Medium/High/Critical ranges), so organizations need to establish their own thresholds. Additionally, the formula's multiplicative nature means edge cases—like a model with maximum scores across all dimensions—could produce scores that are either inflated or compressed depending on how the factors interact. The lack of real-world case studies showing AIVSS scores for known AI incidents (like the Microsoft Tay chatbot failure or Clearview AI's data scraping) makes it hard to calibrate intuition for what different scores actually mean in practice.

Verdict

Use AIVSS if you're building an AI security program from scratch and need a comprehensive methodology to standardize risk assessment across different model types and deployment contexts. It's particularly valuable for regulated industries (finance, healthcare, critical infrastructure) where you need to document why certain AI systems receive more security investment than others, or for organizations with multiple AI teams that need a common language for discussing vulnerability severity. The framework shines when you have dedicated AI security engineers who can perform manual assessments and translate findings into organizational risk registers. Skip AIVSS if you need automated vulnerability scanning, lack personnel with deep AI security expertise, or want plug-and-play tooling. If your primary need is securing LLM applications specifically (rather than broader AI systems), start with the more actionable OWASP Top 10 for LLM Applications, which provides concrete mitigation guidance. And if you're looking for offensive security tools to actually test your models, explore Microsoft's Counterfit or IBM's Adversarial Robustness Toolbox rather than a scoring framework. AIVSS is a governance tool, not a testing tool—know which you need before investing time here.