Building Knowledge Graphs from Conversations: Inside MindGraph's Schema-Driven Architecture

Hook

What if you could build a knowledge graph by just talking to it? MindGraph proves this isn't science fiction—it's a 400-line Flask app that turns conversations into interconnected data structures.

Context

Knowledge graphs power everything from Google's search results to fraud detection systems, but building them traditionally requires armies of data engineers manually defining entities, relationships, and ontologies. The promise of large language models was supposed to change this—AI could extract structured knowledge from unstructured text automatically. The reality? Most implementations still require rigid schemas, complex ETL pipelines, and significant engineering overhead.

MindGraph takes a different approach. Created by Yohei Nakajima (known for his work on BabyAGI), it's a proof-of-concept that demonstrates how schema-driven prompting can guide LLMs to consistently extract graph-structured data from natural language. Rather than building another heavyweight framework, MindGraph explores a fundamental question: what's the minimal architecture needed to let AI populate and query a knowledge graph through conversation? The answer is surprisingly elegant—a Flask API, an in-memory graph store, and a JSON schema that teaches the LLM what your graph should look like.

Technical Insight

The brilliance of MindGraph lies in how it constrains LLM outputs using a declarative schema. At the heart of the system is schema.json, which defines entity types, relationship types, and their properties. This isn't just documentation—it's actively injected into prompts to guide the AI's output format.

Here's how the schema shapes LLM behavior. When you send natural language to MindGraph, the integration system wraps it with schema context:

# From the integration processor
def process_with_schema(user_input, schema):
    prompt = f"""
    Given this schema for a knowledge graph:
    {json.dumps(schema, indent=2)}
    
    Extract entities and relationships from this text:
    {user_input}
    
    Return JSON with:
    - entities: [{"id": "unique_id", "type": "from_schema", "properties": {}}]
    - relationships: [{"from": "entity_id", "to": "entity_id", "type": "from_schema"}]
    """
    
    response = openai.ChatCompletion.create(
        model="gpt-4",
        messages=[{"role": "user", "content": prompt}]
    )
    
    return json.loads(response.choices[0].message.content)

The schema acts as a contract between your domain model and the AI. Say you're tracking tech companies and their funding. Your schema defines entity types like Company, Investor, and Person, plus relationship types like FUNDED_BY and FOUNDED_BY. When you tell MindGraph "Acme Corp raised $10M from Sequoia, founded by Jane Doe," the LLM doesn't hallucinate random entity types—it maps the input to your predefined schema, creating structured nodes and edges.

The graph itself lives in a simple Python class that maintains dictionaries for nodes and edges. This in-memory approach means blazing-fast reads and writes without database overhead:

class GraphStore:
    def __init__(self):
        self.entities = {}  # id -> {type, properties}
        self.relationships = []  # [{from, to, type}]
    
    def add_entity(self, entity_id, entity_type, properties):
        self.entities[entity_id] = {
            'type': entity_type,
            'properties': properties
        }
    
    def add_relationship(self, from_id, to_id, rel_type):
        self.relationships.append({
            'from': from_id,
            'to': to_id,
            'type': rel_type
        })
    
    def query(self, natural_language_query):
        # Convert NL query to graph traversal using LLM
        # Returns matching subgraph
        pass

The plugin architecture deserves special attention. MindGraph scans the integrations/ directory for Python modules and automatically registers any functions decorated with @integration. This lets you add new AI-powered capabilities without touching core code:

# integrations/summarizer.py
from app import integration_manager

@integration_manager.register
def summarize_entity(entity_id):
    """Generate a natural language summary of an entity and its connections."""
    entity = graph.get_entity(entity_id)
    connections = graph.get_relationships(entity_id)
    
    prompt = f"Summarize this entity and its relationships: {entity}, {connections}"
    # Call LLM, return summary

Once registered, this becomes available at /api/integration/summarize_entity. This pattern makes MindGraph incredibly extensible—you can add entity merging, duplicate detection, or custom query languages as plugins without architectural changes.

The querying system is particularly clever. Rather than implementing a query language like Cypher, MindGraph uses the LLM to translate natural language into graph traversals. Ask "Who funded companies in San Francisco?" and the LLM generates a traversal plan: find entities of type Company with property location: San Francisco, then traverse incoming FUNDED_BY relationships. The system executes this plan against the in-memory graph and returns results. It's slower than compiled queries but requires zero query language knowledge from users.

Gotcha

The elephant in the room is persistence—or rather, the complete lack of it. Restart the Flask server and your entire knowledge graph vanishes. This isn't an oversight; it's an intentional design choice for a proof of concept. But it means MindGraph is fundamentally unsuitable for any scenario where you need to preserve data between sessions. You can't build a production application on a foundation that forgets everything when it crashes.

The single-tenant architecture creates additional constraints. There's one global graph shared by all API clients, with no concept of users, workspaces, or access control. In a prototype, this is fine. In any multi-user scenario, it's a security nightmare. You'd need to completely redesign the storage layer to support isolation. The in-memory approach also means your graph size is capped by available RAM, and there's no way to distribute the load across multiple servers. LLM costs and latency present another practical hurdle. Every query hits OpenAI's API, which means response times measured in seconds and costs that scale linearly with usage. For exploration and demos, this works. For a high-traffic application serving thousands of queries, your API bill would be astronomical and users would rage-quit waiting for responses.

Verdict

Use if: You're prototyping a conversational interface to knowledge graphs, researching how to structure LLM prompts for consistent graph extraction, or need a teaching example of schema-driven AI systems. MindGraph excels as a learning resource and template for building custom solutions. It's also perfect for hackathons, demos, or proof-of-concepts where you need to quickly show how natural language can populate a graph. Skip if: You need production reliability, data persistence, multi-user support, or query performance under load. MindGraph is explicitly a prototype, and treating it as production-ready infrastructure will end in tears. For real applications, start with Neo4j or Memgraph and layer LLM integrations on top rather than building from this foundation.