FreeLLMAPI: A 7.4-Billion-Token Proxy That Aggregates 34 Free LLM Providers Behind One Endpoint
Hook
Every month, 7.4 billion tokens flow through free-tier LLM APIs that most developers never touch. FreeLLMAPI turns that sprawl into a single OpenAI-compatible endpoint with automatic failover.
Context
The explosion of free LLM inference tiers—Gemini's 1,500 requests per day, Groq's 14,400 tokens per minute, Cerebras's perpetual beta credits—created a paradox for individual developers. On one hand, you could theoretically run thousands of experiments without spending a dollar. On the other, managing 34 different API formats, rate limits, and provider quirks meant you'd spend more time writing adapter code than building. Tools like LiteLLM offered normalization but required manual YAML configuration for every provider and didn't track rate limits across keys. OpenRouter solved the UX problem but charged for convenience. What was missing was a self-hosted gateway that automatically discovered new free endpoints, learned their quota ceilings from runtime behavior, and rotated through providers intelligently—all while maintaining wire-level compatibility with the OpenAI SDK ecosystem that most agent frameworks expect.
FreeLLMAPI fills that gap with a TypeScript reverse proxy that treats free-tier aggregation as a cataloging problem. Instead of hardcoding provider logic, it fetches a signed JSON feed twice daily from freellmapi.co containing metadata for 635 model endpoints: context windows, rate limits, capability flags, and adapter hints. The router builds a live budget model by watching 429 responses and provider-specific headers, then ranks candidates per request based on remaining quota, cached success rates, and optional sticky-session constraints. For developers running coding agents like Cline or Cursor, this means pointing OPENAI_BASE_URL at localhost and forgetting about which provider is handling the request—until you hit a legal problem.
Technical Insight
The core architecture is a tiered routing engine with per-provider adapters that normalize distinct API shapes into OpenAI's wire format. When a request hits /v1/chat/completions, the router doesn't immediately pick a provider. It first evaluates your request parameters—model name, context length, streaming preference—against the cached catalog, filtering to compatible endpoints. Then it checks per-(provider, model, key) rate counters stored in SQLite, eliminating any that have exhausted their RPM or TPD budgets. What remains is a ranked chain of candidates sorted by success rate from the last 100 requests, with an optional sticky-session override if you've got session_id in your headers.
Here's what the adapter layer looks like for a Gemini request normalization:
// Simplified from the actual Gemini adapter
interface OpenAIMessage {
role: 'system' | 'user' | 'assistant';
content: string;
}
interface GeminiContent {
role: 'user' | 'model';
parts: Array<{ text: string }>;
}
function toGeminiFormat(messages: OpenAIMessage[]): GeminiContent[] {
// Gemini doesn't have a 'system' role—merge into first user message
const systemPrompts = messages
.filter(m => m.role === 'system')
.map(m => m.content)
.join('\n\n');
const conversational = messages.filter(m => m.role !== 'system');
return conversational.map((msg, idx) => ({
role: msg.role === 'assistant' ? 'model' : 'user',
parts: [{
text: idx === 0 && systemPrompts
? `${systemPrompts}\n\n${msg.content}`
: msg.content
}]
}));
}
This pattern repeats for Anthropic (which uses messages arrays with role: 'user' | 'assistant' but requires a top-level system parameter), Mistral (which expects messages but has different function-calling schemas), and 31 other providers. The adapters live in src/providers/, and the catalog update mechanism lets the maintainer fix quirks—like Groq's recent breaking change to tool-call IDs—without requiring users to pull and redeploy.
The rate-limit learning is where things get interesting. Instead of trusting provider documentation, FreeLLMAPI watches responses. When a 429 arrives, it parses Retry-After headers and provider-specific budget headers (Gemini sends x-ratelimit-remaining-requests, OpenRouter sends x-ratelimit-limit-tokens), then updates an in-memory budget model. For providers that don't expose headers, it falls back to exponential backoff with a learned ceiling—if three consecutive requests to groq/llama3-8b fail in a minute, the router assumes you've hit the RPM cap and deprioritizes that (provider, model) tuple for 60 seconds.
Here's the simplified budget check:
function selectProvider(request: ChatRequest, catalog: ModelCatalog): Provider | null {
const compatible = catalog.models.filter(m =>
m.name === request.model &&
m.maxTokens >= estimateTokens(request.messages)
);
for (const model of compatible) {
const budget = rateLimits.get(`${model.provider}:${model.name}`);
if (!budget || budget.remainingRPM > 0) {
return model.provider;
}
}
return null; // All providers exhausted
}
The sticky-session logic adds a session_id header that pins a conversation thread to one provider for 30 minutes. This prevents mid-chat persona drift when you're talking to a coding agent and the router fails over from Claude-via-OpenRouter to Gemini-via-Vertex. If the sticky provider dies, the router emits a compact handoff note—last two messages plus a system prompt explaining the context—so the new model can pick up the thread.
The self-updating catalog is the architectural linchpin. Every 12 hours, the router fetches a signed JSON feed from https://catalog.freellmapi.co/models.json containing:
{
"version": "2024-01-15T08:00:00Z",
"models": [
{
"name": "gemini-2.0-flash",
"provider": "google-vertex",
"maxTokens": 1000000,
"rpmLimit": 1500,
"tpdLimit": null,
"capabilities": ["streaming", "function-calling"],
"adapterHints": { "systemRoleHandling": "merge-to-user" }
}
],
"signature": "ed25519:a3f2..."
}
This separation means quota changes, new free models, and provider quirk fixes propagate without code deploys. The signature prevents tampering, but it also creates a single point of control—if freellmapi.co disappears, your router stops updating. The premium tier gets same-day catalog updates; the free open-source version gets a 30-day-delayed snapshot, which is a clever monetization lever that doesn't break the MIT license.
For multi-model synthesis, the virtual fusion model fans out parallel requests to three diverse endpoints (typically a reasoning model, a fast model, and a creative model), then passes all responses to a judge model with a prompt like "Which answer best addresses the user's question?" This burns quota fast—one user request becomes four provider requests—but it's architecturally interesting as a built-in ensembling primitive that most gateways don't offer.
Gotcha
The 'personal experimentation only' disclaimer in the README is load-bearing legal language. Aggregating 34 free tiers at any meaningful scale violates every provider's terms of service. Google's Vertex AI free tier explicitly forbids commercial use and automated abuse; Groq's terms prohibit 'excessive or abusive usage patterns'; OpenRouter's free endpoints come with rate limits designed to prevent resale. If you scale this beyond personal tinkering—say, exposing it to a team or embedding it in a SaaS product—you risk simultaneous bans from every provider in the chain, and you have no recourse because you were never a paying customer. This isn't a hypothetical: developers have reported account terminations after routing production traffic through similar aggregators.
The security model also assumes you're the only user on a trusted machine. API keys are AES-256-GCM encrypted in SQLite, but the encryption key lives in your .env file on the same filesystem. An attacker with read access gets both the ciphertext and the key, rendering the encryption theater. The unified bearer token for your FreeLLMAPI instance is transmitted in plaintext over HTTP unless you manually set up a reverse proxy with TLS, which most local development setups skip. There's no multi-tenancy, no request isolation, no audit logging—if you expose this to a network, you're handing out your provider keys to anyone who guesses the bearer token. The catalog update mechanism is also a trust bottleneck: you're executing adapter logic and routing rules fetched from a third-party server, and while the signature prevents tampering, it doesn't prevent the catalog maintainer from inserting malicious model configs that exfiltrate prompts or inject system messages.
Verdict
Use FreeLLMAPI if you're an individual developer running coding agents (Cline, Cursor, Aider) for personal projects and you're allergic to paying for tokens. The self-updating catalog and sticky-session logic will save you hours of YAML wrangling compared to LiteLLM, and the broad client compatibility means you can point almost any tool at it without rewriting integration code. It's particularly strong for experimentation workflows where you're iterating fast on prompts and don't care which model answers, as long as something does. Skip it entirely if you need production reliability, multi-tenancy, compliance, or any scenario where getting banned by 34 providers simultaneously would be catastrophic. Don't expose this to a team, don't embed it in a product, and don't expect it to handle more than a few hundred requests per day without hitting free-tier ceilings. If you're willing to pay $5/month for peace of mind, just use OpenRouter—you'll get better rate limits, actual ToS compliance, and someone to email when things break.