Auto-Ontology: NVIDIA's Graph-Based Semantic Layer for Deterministic LLM Query Generation
Hook
Every text-to-SQL system eventually generates a query that looks perfect, executes successfully, and returns completely wrong results. NVIDIA's Auto-Ontology attacks this problem by making 'I don't know' a first-class response.
Context
The promise of natural language database queries has haunted the software industry since the 1970s. Today's LLMs can generate syntactically valid SQL from plain English with shocking consistency, but syntax correctness is a dangerously low bar. When a business analyst asks 'What were Q3 sales in the Northeast region?' and your LLM joins the wrong tables, applies filters to columns that don't contain what their names suggest, or fabricates metrics that sound plausible, you've automated the production of confident incorrectness at scale.
The traditional solution—throwing more training data at the model—treats the symptom, not the disease. The real problem is that relational schemas are semantic minefields. Column names lie. Foreign keys go undeclared. The same business concept scatters across six tables with three different naming conventions. Bridge tables connect entities through implicit many-to-many relationships that exist only in data patterns, not metadata. What's missing isn't a better language model; it's a machine-readable representation of what the database actually means. NVIDIA's Auto-Ontology builds that semantic layer as a queryable graph, then uses it to constrain LLM behavior—not through prompt engineering, but through architectural guarantees that make hallucinated queries structurally impossible.
Technical Insight
Auto-Ontology implements semantic-layer construction as a three-phase pipeline that feels more like a compiler than a typical LLM application. The ingestion phase extracts schemas, profiles column distributions, samples actual values, and generates embeddings for every semantic entity it discovers. These embeddings land in pgvector alongside the raw catalog metadata, creating a foundation for retrieval-augmented generation that operates on statistical properties of your actual data, not just table names.
The compilation phase is where things get interesting. The system performs entity extraction table-by-table, building a graph of Terms (business concepts like 'Customer' or 'Revenue'), Relationships (how tables join, including automatic bridge table detection), and SQL Attributes (certified metric expressions that capture tribal knowledge like 'active users means login_count > 0 in the last 30 days'). Bridge table inference doesn't just read foreign key constraints—it analyzes data patterns to discover many-to-many relationships even when your schema lacks explicit junction tables. This produces a graph structure where edges represent not just technical joins but semantic relationships the database actually supports.
Query generation inverts the typical text-to-SQL flow. Instead of schema → prompt → SQL, you get question → answerability check → candidate retrieval → generation → validation. Here's what the answerability primitive looks like in practice:
# The check_answerable tool computes coverage before attempting SQL
result = mcp_client.call_tool(
"check_answerable",
{"question": "What were Q3 sales in the Northeast region?"}
)
# Returns structured coverage analysis:
{
"answerable": true,
"coverage_score": 0.87,
"missing_concepts": [],
"available_terms": ["sales", "quarter", "region"],
"suggested_clarifications": []
}
# If coverage is low, you get specifics:
{
"answerable": false,
"coverage_score": 0.23,
"missing_concepts": ["customer_lifetime_value"],
"available_terms": ["total_purchases", "customer_tenure"],
"suggested_clarifications": [
"Did you mean total purchase value? We don't track CLV."
]
}
This coverage computation happens via embedding-based retrieval against the ontology graph, using separate models for indexing (nemotron-3-embed-1b during ingestion) and reranking (llama-nemotron-rerank-vl-1b-v2 at query time). The two-stage approach trades latency for precision—initial retrieval casts a wide net, reranking scores candidates against the actual question semantics. When coverage exceeds the threshold, the system retrieves relevant Terms and Relationships, then generates SQL constrained by the ontology's explicit join paths.
The validation step is equally critical. Before executing generated SQL, the system checks filter values against actual database contents:
# If the query contains WHERE region = 'Northeast', validation:
# 1. Identifies the region column in the ontology
# 2. Queries distinct values from the actual table
# 3. Fuzzy-matches 'Northeast' against real values
# 4. Rejects or rewrites if no match exists
# This prevents queries like:
SELECT SUM(amount) FROM sales WHERE region = 'Northeast'
# when your database actually uses 'NE' or 'North East' or region_id = 3
SQL Attributes deserve special attention because they solve the 'tribal knowledge' problem. When a data analyst writes a complex query to calculate a business metric, that logic typically dies in a Slack thread or gets copy-pasted with subtle variations across notebooks. Auto-Ontology stores these as certified expressions in the ontology:
-- Instead of embedding this logic in prompts:
CREATE ATTRIBUTE monthly_active_users AS (
SELECT
DATE_TRUNC('month', login_timestamp) AS month,
COUNT(DISTINCT user_id) AS mau
FROM user_sessions
WHERE login_timestamp >= CURRENT_DATE - INTERVAL '30 days'
GROUP BY 1
)
When someone asks about MAU, the LLM doesn't reinvent this calculation—it references the certified attribute by name. The ontology becomes a versioned, reviewable business logic layer that survives schema migrations and analyst turnover. Query history analysis feeds back into ontology suggestions, creating a flywheel where common patterns automatically promote to certified metrics.
The architecture makes strong opinions about trust boundaries. The Next.js frontend (port 3000) is the only public-facing component. It handles authentication via Better Auth, then forwards requests to the FastAPI backend (3001) and ingestion worker (3002) with an x-auto-ontology-user-id header. Internal services trust this header implicitly—there's no internal authentication. This means network isolation isn't optional; it's load-bearing. Expose port 3001 to the internet and you've created an authentication bypass where anyone can impersonate any user by setting a header. For internal deployments behind a VPN, this trades off security complexity for operational simplicity. For multi-tenant SaaS, it's a non-starter.
Gotcha
The deployment model assumes you control the network perimeter and can treat Postgres as a single point of failure for both application state and semantic metadata. There's no ontology federation—if your data estate spans Snowflake, BigQuery, and on-prem warehouses, you'll run separate Auto-Ontology instances with separate ontology graphs, and cross-database queries aren't supported. The system models only relational joins, so document stores, GraphQL APIs, and key-value databases are out of scope entirely.
Embedding model lock-in creates operational friction. The EMBED_MODEL setting controls vector generation during ingestion, and switching models requires full re-ingestion to rebuild the pgvector index. There's no migration path for A/B testing retrieval quality improvements—you either commit to the cutover and reprocess your entire catalog, or maintain parallel deployments. For organizations with hundreds of tables and terabytes of profiling data, this re-ingestion window can measure in days. The MCP server provides read-only access for external agents, but ontology refinement requires direct database access or UI interaction. You can't programmatically update SQL Attributes, add Terms, or modify Relationships through the API, which limits CI/CD integration for semantic layer governance. Teams expecting to version-control their ontology as code will need to bolt on custom tooling or manipulate Postgres tables directly.
Verdict
Use Auto-Ontology if you're running LLM agents over complex relational warehouses (50+ tables) where answer correctness matters more than sub-second latency, your data team wastes hours answering the same business questions phrased differently, and you can deploy behind a controlled network perimeter. The bridge table inference and answerability checking are genuinely novel, and the SQL Attributes layer solves real tribal knowledge problems that dbt's YAML-based semantic layer doesn't address. Skip if you're building multi-tenant SaaS (the header-trust auth model won't work), need to federate ontologies across heterogeneous data stores (Postgres coupling is mandatory), or your data lives primarily in non-relational systems. The three-phase compilation pipeline is the differentiator—if you just need basic text-to-SQL, lighter frameworks like LlamaIndex with SQLDatabase will get you there faster.