🔴 Advanced ⚙️ Type: Vision-Language Model / Document Parsing 💸 Free & Open Source (MIT) ⭐ 5,200+ GitHub Stars
What is Unlimited-OCR?
Unlimited-OCR is a groundbreaking document parsing model developed by Baidu that officially ushers in the era of “One-shot Long-horizon Parsing.” If you have ever tried to use an AI to read a massive 50-page PDF, you likely ran into the classic VRAM bottleneck: the AI’s short-term memory (KV cache) fills up, and the application crashes. To bypass this, developers traditionally have to slice documents page-by-page, process them individually, and stitch the text back together.
Unlimited-OCR eliminates this problem. By combining the ultra-high compression rate of DeepSeek-OCR’s encoder with a novel architecture called Reference Sliding Window Attention (R-SWA), the model can transcribe dozens of document pages in a single, continuous forward pass.
How does it work? R-SWA splits the AI’s focus. It maintains a Global Reference of the document images so it never loses context, but it applies a strict Sliding Window to its own generated text. It safely “forgets” the text it typed a few paragraphs ago while keeping its eyes firmly on the image. This locks the KV cache at a constant size, allowing you to feed it theoretically unlimited pages without crashing your GPU.
Who is it for?
- Data Scientists and AI Engineers building Enterprise RAG (Retrieval-Augmented Generation) pipelines who need to quickly digitize massive, multi-page financial reports or scientific papers with complex layouts.
- Developers using SGLang who want a blazing-fast, OpenAI-compatible local API endpoint to handle bulk document processing.
- Archivists and Researchers looking for an open-source, local alternative to costly proprietary OCR APIs (like Google Cloud Vision or AWS Textract) to digitize Chinese and English documents.
What makes it special?
- One-Shot Multi-Page Inference — You do not need to write complex orchestration loops. You simply hand the model an array of image files (or a raw PDF), and it reads the entire sequence in one continuous flow up to a 32K context window.
- Native PDF Support — Using the included PyMuPDF integration, you can pass a standard
.pdffile directly into the generation script. It converts the pages to high-DPI image tiles and parses them instantly. - Blazing Fast SGLang Backend — While you can run it via standard Hugging Face Transformers, the repository is heavily optimized to run on SGLang, utilizing FlashAttention-3 and custom logit processors to maximize token throughput.
- Open Weights — The model weights are hosted publicly on Hugging Face (
baidu/Unlimited-OCR), allowing complete local deployment without data privacy concerns.
Requirements before you start
Unlimited-OCR is a heavy-duty Vision-Language Model. You cannot run this on a standard laptop CPU; it requires serious AI hardware.
- NVIDIA GPU — You need a modern CUDA-capable GPU with ample VRAM (e.g., RTX 3090/4090 or datacenter GPUs like A100/H100) to host the model weights.
- CUDA 12.x — Specifically, the developers recommend CUDA 12.9 with Python 3.12+.
- PyTorch & SGLang — Required for running the high-performance API server.
- Language Support: Currently optimized heavily for English and Chinese documents.
Step-by-step installation
The developers highly recommend running Unlimited-OCR via an SGLang server rather than standard Transformers, as it is significantly faster and handles the memory caching much better.
Step 1 — Set Up the Environment
Create a fresh virtual environment and install the required dependencies (assuming you already have PyTorch and CUDA configured):
pip install sglang[all]
pip install pymupdf requestsStep 2 — Launch the SGLang Server
Open a terminal and start the SGLang backend. This command will automatically download the model weights from Hugging Face and expose an OpenAI-compatible API on port 10000.
python -m sglang.launch_server \
--model baidu/Unlimited-OCR \
--served-model-name Unlimited-OCR \
--attention-backend fa3 \
--page-size 1 \
--mem-fraction-static 0.8 \
--context-length 32768 \
--enable-custom-logit-processor \
--disable-overlap-schedule \
--skip-server-warmup \
--host 0.0.0.0 \
--port 10000Step 3 — Write a Python Script to Send a PDF
Leave the server running and open a new terminal or Python file. You can now write a script that converts a multi-page PDF into images and sends it to your local API for one-shot transcription:
import base64
import fitz # PyMuPDF
import requests
import tempfile
import os
# Helper function to convert PDF pages to images
def pdf_to_images(pdf_path, dpi=300):
doc = fitz.open(pdf_path)
tmp_dir = tempfile.mkdtemp(prefix='pdf_ocr_')
mat = fitz.Matrix(dpi / 72, dpi / 72)
paths = []
for i, page in enumerate(doc):
out = os.path.join(tmp_dir, f'page_{i+1:04d}.png')
page.get_pixmap(matrix=mat).save(out)
paths.append(out)
doc.close()
return paths
# Helper to encode images
def encode_image(image_path):
with open(image_path, "rb") as f:
data = base64.b64encode(f.read()).decode("utf-8")
return {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{data}"}}
# Run inference
pdf_images = pdf_to_images("your_document.pdf")
payload = {
"model": "Unlimited-OCR",
"messages": [
{"role": "user", "content": [{"type": "text", "text": "Multi page parsing."}] + [encode_image(p) for p in pdf_images]}
],
"max_tokens": 32768
}
response = requests.post("http://127.0.0.1:10000/v1/chat/completions", json=payload)
print(response.json()['choices'][0]['message']['content'])Common errors and fixes
| Error | What it means | How to fix it |
|---|---|---|
CUDA Out of Memory (OOM) during long PDF parsing | Despite the R-SWA memory improvements, feeding a 100-page PDF at high DPI into a standard desktop GPU can still exceed VRAM limits. | Lower the DPI setting in your pdf_to_images() function (e.g., from 300 to 150), or reduce the --mem-fraction-static flag on your SGLang server launch command. |
SGLang Server fails to start: fa3 backend not found | You told the server to use FlashAttention-3 (--attention-backend fa3), but the package is not installed or your GPU architecture is too old to support it. | Ensure you have a modern GPU (Hopper or Ada Lovelace). If you are on an older GPU (like an RTX 3090), remove the --attention-backend fa3 flag from the launch command and let SGLang fall back to standard FlashAttention-2. |
| Model endlessly repeats the same sentence (Hallucination loop) | The generative decoder lost context and got stuck in a repetitive text loop—a common issue with long-horizon VLMs. | This is why the developers implemented the custom logit processor. Ensure you are launching the SGLang server with the --enable-custom-logit-processor flag, which forces an n-gram repetition penalty to break these loops. |
Free vs Paid comparison
| Feature | Baidu Unlimited-OCR (Local) | Cloud OCR (e.g., AWS Textract / Azure Form Recognizer) |
|---|---|---|
| Cost per Page | $0 (You pay only for your local electricity) | $0.01 to $0.05 per page |
| Data Privacy | ✅ 100% Secure (No data leaves your GPU) | ❌ Documents uploaded to enterprise cloud servers |
| Context Horizon | 🟢 Excellent (Reads whole documents holistically) | 🔴 Fragments context page by page |
| Setup & Hardware Requirements | 🔴 Requires expensive high-VRAM GPUs | 🟢 Zero local hardware needed |
Bottom line: Unlimited-OCR is a massive engineering leap for document processing. Slicing PDFs into single pages destroys context—if a data table spans across page 3 and page 4, traditional OCR breaks. By utilizing R-SWA to stabilize the KV cache, Baidu has open-sourced a model that can ingest entire booklets natively. If your company processes thousands of multi-page invoices or reports daily, deploying an SGLang server with this model will save you thousands of dollars in cloud API fees.
Alternatives — 3 similar tools
1. DeepSeek-VL / OCR
The foundational model that Unlimited-OCR actually builds upon. DeepSeek’s vision models are famously efficient, utilizing highly compressed encoders. If you do not need the specific “infinite multi-page” architecture that Baidu introduced and just want raw, accurate parsing for single complex images or charts, the native DeepSeek models are fantastic.
2. Marker (by Surya)
If you don’t have a massive GPU and need a lightweight, highly accurate PDF-to-Markdown parser, Marker is the current open-source king. Instead of relying purely on heavy generative VLMs, Marker uses a combination of traditional heuristic parsing and lightweight deep learning models to extract text, equations, and tables from PDFs incredibly fast.
🔗 github.com/VikParuchuri/marker
3. Qwen2-VL
Alibaba’s state-of-the-art vision model. Qwen2-VL is renowned for its incredible performance in OCR, chart reading, and visual reasoning. While it generally processes images frame-by-frame (or page-by-page) rather than utilizing the continuous sliding window of Unlimited-OCR, its base accuracy in identifying dense text is arguably among the best in the open-source world.
🚀 Want more free AI tools like this?
We find, test, and write setup guides for the best free and open-source AI tools — so you don’t have to dig through GitHub yourself.Browse Free AI Tools at globalaiforce.com/shop →
📸 Follow us for daily AI tool tips and tutorials: instagram.com/globalaiforce