Building Search-Augmented Chat with Exa and o3-mini: A Template for Real-Time Web Context

Hook

Most chatbots hallucinate because they're trained on stale data. What if your LLM could search the web like a human researcher before answering?

Context

The fundamental limitation of language models is their training cutoff date. Ask GPT-4 about last week's tech news, and you'll get an apologetic "I don't have information about that." Retrieval-Augmented Generation (RAG) has emerged as the standard solution, but most implementations rely on static document collections or clunky keyword-based search APIs that return irrelevant results.

Exa-o3mini-chat represents a newer approach: pairing neural search with reasoning-optimized language models. Exa's API uses neural embeddings to understand semantic intent rather than just matching keywords, while OpenAI's o3-mini model brings enhanced reasoning capabilities to synthesize search results into coherent answers. This combination creates chat experiences that feel current and grounded without the hallucination problem that plagues pure LLM approaches. The repository serves as a production-ready template built on Next.js 14 and the Vercel AI SDK, demonstrating how to wire these APIs together with minimal boilerplate.

Technical Insight

The architecture follows a clean separation between frontend chat interface and API route handlers. The magic happens in the server-side route that orchestrates the search-then-generate workflow. When a user submits a query, the application doesn't immediately hit o3-mini. Instead, it first calls Exa's search API to fetch relevant, recent web content, then passes that context as additional information to the language model.

Here's how the core API route structures this workflow:

import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
import Exa from 'exa-js';

const exa = new Exa(process.env.EXA_API_KEY);

export async function POST(req: Request) {
  const { messages } = await req.json();
  const lastMessage = messages[messages.length - 1].content;

  // First, search the web for relevant context
  const searchResults = await exa.searchAndContents(lastMessage, {
    numResults: 5,
    useAutoprompt: true,
    text: { maxCharacters: 1000 }
  });

  // Format search results into context string
  const context = searchResults.results
    .map(r => `Title: ${r.title}\nURL: ${r.url}\nContent: ${r.text}`)
    .join('\n\n---\n\n');

  // Augment the conversation with search context
  const augmentedMessages = [
    ...messages.slice(0, -1),
    {
      role: 'system',
      content: `You are a helpful assistant. Use the following web search results to provide accurate, up-to-date information:\n\n${context}`
    },
    messages[messages.length - 1]
  ];

  // Stream the response from o3-mini
  const result = await streamText({
    model: openai('o3-mini'),
    messages: augmentedMessages,
  });

  return result.toAIStreamResponse();
}

The useAutoprompt parameter is particularly clever—it lets Exa's API rewrite the user's query into a more search-optimized form, similar to how an experienced researcher would reformulate questions for Google. This improves result quality without forcing developers to write complex prompt engineering logic.

The frontend leverages Vercel AI SDK's useChat hook, which abstracts away the complexity of streaming responses and state management. The developer experience is remarkably clean—you get real-time token streaming, automatic message history, and loading states with just a few lines of React:

import { useChat } from 'ai/react';

export default function ChatInterface() {
  const { messages, input, handleInputChange, handleSubmit, isLoading } = useChat({
    api: '/api/chat'
  });

  return (
    <div className="flex flex-col h-screen">
      <div className="flex-1 overflow-y-auto">
        {messages.map(m => (
          <div key={m.id} className={m.role === 'user' ? 'text-right' : 'text-left'}>
            {m.content}
          </div>
        ))}
      </div>
      <form onSubmit={handleSubmit}>
        <input value={input} onChange={handleInputChange} disabled={isLoading} />
      </form>
    </div>
  );
}

The pattern here is intentionally minimal. Unlike heavyweight frameworks like LangChain that introduce multiple abstraction layers, this approach gives you direct control over the search-to-generation pipeline. You can easily modify which fields Exa returns, adjust the number of search results, or change how context is formatted before being passed to the LLM. The tradeoff is less built-in functionality—there's no conversation memory management beyond what the frontend provides, no automatic fallback strategies if APIs fail, and no sophisticated prompt chaining.

One architectural decision worth noting: the application passes search results as a system message rather than using o3-mini's function calling capabilities. This simplifies the implementation but means you lose the ability to have the model decide whether to search at all. Every query triggers a search, which impacts latency and API costs. A more sophisticated version might use tool calling to let o3-mini determine when web context is actually necessary.

Gotcha

The dual API dependency creates both cost and reliability concerns. Every single chat message incurs charges from both Exa and OpenAI, and there's no caching layer mentioned in the codebase. If a user asks the same question twice, you'll pay for duplicate searches and duplicate LLM calls. For a prototype, this is fine. For production, you'd need to implement Redis caching or a similar strategy to memoize search results.

The 43-star count reflects the repository's early stage. There's minimal error handling—if Exa's API is down or rate-limits you, the application will simply fail with a generic error. The README provides setup instructions but doesn't cover deployment considerations like environment variable management, rate limiting to prevent abuse, or monitoring API costs. The code also lacks TypeScript types for Exa's search results, which means you lose compile-time safety when accessing result properties. You'll need to add Zod schemas or similar validation if you want production-grade type safety. Additionally, o3-mini's pricing model is still relatively new, and its reasoning capabilities come with higher latency compared to GPT-3.5-turbo or GPT-4-turbo. Users expecting instant responses may find the delay noticeable.

Verdict

Use if: You're building a prototype that needs current web information and you want to avoid the complexity of setting up vector databases and embedding pipelines. This template gets you from zero to functional search-augmented chat in under an hour, and the minimalist architecture makes it easy to understand and modify. It's also ideal if you're specifically interested in evaluating Exa's neural search capabilities or o3-mini's reasoning performance in a real application context. Skip if: You need production-grade reliability, cost controls, or extensive customization options. The lack of caching, error handling, and monitoring makes this unsuitable for customer-facing applications without significant hardening. Also skip if you want model flexibility—this is tightly coupled to OpenAI's ecosystem, so switching to Anthropic's Claude or open-source alternatives would require substantial refactoring. Consider alternatives like LangChain with Tavily search if you need a more battle-tested framework, or build your own RAG pipeline with Pinecone if you want full control over embeddings and retrieval strategies.