> your AI agent picks dependencies from memory; give it dated facts — try starlog.dev ↗ vet your agent's deps ↗ vibe-coding is fine. vibe-importing isn’t. — try starlog.dev ↗ vibe-importing isn’t fine ↗ your agent has never seen your private packages — try starlog.dev ↗ facts for private packages ↗ a linter for the dependencies your AI agent picks — try starlog.dev ↗ a linter for agent deps ↗ whois is redacted, cdns mask the rest — get the real operator — try whoisgeni.us ↗ who really runs that domain ↗ domain attribution that shows its work — full evidence chain — try whoisgeni.us ↗ domain intel w/ evidence ↗

← Back to Articles

Unlimited-OCR: How Baidu's Dual-Mode Vision Pipeline Solves the OCR Detail-Context Dilemma

[ View on GitHub ]

Unlimited-OCR: How Baidu's Dual-Mode Vision Pipeline Solves the OCR Detail-Context Dilemma

Hook

Most OCR models force you to choose between capturing tiny footnotes and understanding page layout. Unlimited-OCR's answer? Process the same document in two fundamentally different ways depending on what you're after.

Context

Traditional OCR pipelines face a brutal trade-off: downsample a document to fit your model's input resolution and lose fine details like subscripts or footer text, or tile it into crops and sacrifice cross-region context that's critical for understanding tables spanning columns or multi-column layouts. Tesseract handles this with multi-pass heuristics. Modern vision-language models like Nougat try to solve it with transformer attention over the entire page. But when you're dealing with a 300-DPI scan of a dense financial statement—small text, complex tables, watermarks—neither approach gives you both granular accuracy and structural coherence.

Unlimited-OCR, built on DeepSeek-OCR's foundation by Baidu's research team, takes a different path: it gives you two preprocessing modes and makes you choose your poison explicitly. The 'gundam' mode tiles high-resolution images into 640px crops with a 1024px base resolution for detail-obsessed single-page extraction. The 'base' mode resizes everything to 1024x1024 for multi-page consistency. It's pragmatic engineering over algorithmic elegance—acknowledge the trade-off exists, optimize both paths separately, and let deployment context dictate which you use. The model outputs markdown with embedded spatial annotations using special tokens, positioning itself for document intelligence workflows where you need both extracted text and bounding boxes for downstream processing.

Technical Insight

The architecture processes documents through a hybrid pipeline that diverges at the preprocessing stage. In gundam mode, images are tiled into 640px crops after establishing a 1024px base—think of it as a sliding window over your document that preserves DPI-level detail. Base mode simply resizes to 1024x1024, treating the page as a single visual chunk. Both feed into the vision encoder, but the implications cascade through inference.

Here's what deployment looks like with the SGLang backend, which supports custom kernels for Hopper-era GPUs:

import sglang as sgl
from PIL import Image
import numpy as np

# Initialize with overlap disabled—critical for stateful logit processors
llm = sgl.Engine(
    model_path="deepseek-ai/deepseek-ocr",
    tp_size=1,
    trust_remote_code=True,
    disable_overlap=True  # Required for n-gram suppression
)

@sgl.function
def ocr_document(s, image_path, mode="gundam"):
    s += sgl.user(sgl.image(image_path))
    s += sgl.assistant(
        sgl.gen(
            "response",
            max_tokens=32768,
            stop=["<|end|>"],
            sampling_params={"temperature": 0.1}
        )
    )

# Gundam mode for dense single page
state = ocr_document.run(
    image_path="financial_statement.png",
    mode="gundam"
)
markdown_output = state["response"]

The real engineering insight is in how it handles autoregressive pathologies. Vision-language OCR models have a nasty tendency to loop infinitely on visually ambiguous tokens—think repeated characters in tables or watermark patterns that trigger token cycling. Unlimited-OCR implements DeepseekOCRNoRepeatNGramLogitProcessor, a stateful component that suppresses n-gram repetition across sliding windows:

# Simplified conceptual implementation
class DeepseekOCRNoRepeatNGramLogitProcessor:
    def __init__(self, ngram_size=35):
        self.ngram_size = ngram_size
        self.history = []
    
    def __call__(self, input_ids, scores):
        # Track generated n-grams in sliding window
        if len(self.history) >= self.ngram_size:
            recent_ngram = tuple(self.history[-self.ngram_size:])
            # Suppress tokens that would repeat this pattern
            for token_id in range(scores.shape[-1]):
                if self.would_repeat(recent_ngram, token_id):
                    scores[token_id] = float('-inf')
        return scores

This runs during generation, not as post-processing—it's baked into the logit calculation. The ngram_size=35 is tuned for typical OCR token patterns, and the processor maintains state across the entire decode, which is why SGLang's disable_overlap=True is mandatory. Overlapping execution would desynchronize the suppression window.

Multi-page handling concatenates page embeddings rather than processing sequentially. You convert a PDF to images at 300 DPI using PyMuPDF, feed all pages into the vision encoder simultaneously, and let the transformer's attention mechanism handle cross-page references:

import fitz  # PyMuPDF
from transformers import AutoModel, AutoTokenizer

def process_multipage_pdf(pdf_path, mode="base"):
    doc = fitz.open(pdf_path)
    images = []
    
    for page_num in range(len(doc)):
        page = doc[page_num]
        pix = page.get_pixmap(dpi=300)
        img = Image.frombytes("RGB", [pix.width, pix.height], pix.samples)
        images.append(img)
    
    # All pages processed in single context window
    model = AutoModel.from_pretrained(
        "deepseek-ai/deepseek-ocr",
        trust_remote_code=True
    )
    
    # Concatenate embeddings up to 32K token limit
    result = model.generate(
        images=images,
        max_new_tokens=32768,
        mode=mode
    )
    return result

The 32K context limit is the hard ceiling—roughly 30-40 pages depending on content density. Beyond that, you're manually chunking and losing cross-chunk context.

Spatial annotations embed directly in the markdown stream using a custom token format: <|det|>type [x1,y1,x2,y2]<|/det|>. The model generates these during text decoding, which means layout understanding happens inline with language modeling. You extract structure by parsing the output with regex:

import re

DET_RE = re.compile(r'<\|det\|>(\w+)\s+\[([\d,]+)\]<\|/det\|>')

def extract_spatial_annotations(markdown_text):
    annotations = []
    for match in DET_RE.finditer(markdown_text):
        element_type = match.group(1)  # 'table', 'figure', 'text'
        bbox = list(map(int, match.group(2).split(',')))
        annotations.append({
            'type': element_type,
            'bbox': bbox,
            'text': markdown_text[match.start():match.end()]
        })
    return annotations

This is clever reuse of the language model's generation capacity—no separate detection head required—but it's fragile. Malformed output breaks the regex, and there's no fallback mechanism documented in the codebase.

Gotcha

The 32K token context limit isn't a soft ceiling you can work around—it's architectural. A 50-page technical manual will exceed this even with aggressive tokenization, and the codebase provides zero guidance on chunking strategies that preserve coherence across splits. You're on your own for implementing overlap windows or reference resolution between chunks.

Gundam mode's tiling processes crops somewhat independently. While the vision encoder sees the full tiled image, there's no documented overlap or cross-tile attention mechanism for text that spans tile boundaries. In practice, this means a table cell split across tiles might get duplicated or truncated in the output, and you'll only discover this through empirical testing on your specific document types.

The PDF rasterization requirement is a fundamental limitation masquerading as a preprocessing step. PyMuPDF converts everything to 300 DPI images, which destroys native text layers in born-digital PDFs. If your documents are generated from LaTeX or Word—not scanned—you're throwing away lossless text data and reintroducing OCR errors unnecessarily. Tesseract with direct PDF text extraction would be both faster and more accurate for this use case.

No quantization support is documented despite vLLM and SGLang integration. Running fp16 inference on a vision-language model of this size realistically requires 80GB A100s or H100s. The economics don't work for most teams—you're looking at $3-5 per million tokens on cloud GPU providers, versus pennies for quantized alternatives or traditional OCR. The README's timeline references dates in 2026 while being released in 2024/2025, which either indicates documentation sloppiness or suggests this is a research preview rushed to release. Either way, it undermines confidence for production deployment until the community validates the claims independently.

Verdict

Use if: You're processing dense single-page documents (financial statements, academic papers, technical diagrams) where gundam mode's tiling will capture fine text that 1024px resizing would lose, you need spatial annotations with bounding boxes for downstream document intelligence workflows, you have access to 80GB GPUs or can justify the cloud inference costs, and your documents stay under 30-40 pages to fit the 32K context window. The dual-mode preprocessing genuinely solves a real problem that one-size-fits-all models don't handle well. Skip if: Your documents exceed 40 pages (the context limit will force you into manual chunking that negates the model's coherence advantages), you're processing born-digital PDFs where native text extraction would be lossless (you're paying GPU costs to rasterize and re-OCR perfectly good text), you need quantized deployment for cost reasons (no 4-bit or 8-bit support means you're stuck with expensive fp16), or you're looking for proven production stability (the timeline inconsistencies and lack of community validation suggest this might be a research artifact). For long-form documents, DeepSeek-OCR-2 without the tiling complexity might serve you better. For born-digital PDFs, Tesseract plus LayoutParser will be 100x cheaper and potentially more accurate.