🟡 Intermediate ⚙️ Type: OCR / Document AI 💸 Free & Open Source ⭐ 20,800+ GitHub Stars
What is Surya?
Surya is a high-performance, multilingual Optical Character Recognition (OCR) toolkit that gives AI “universal vision.” Developed by Datalab, it is designed to flawlessly read and extract structured text from highly complex documents across more than 90 languages.
Most traditional OCR engines simply dump raw, unformatted text onto your screen. Surya goes much further by performing comprehensive layout analysis. It can look at a dense PDF and instantly detect tables, column structures, images, headers, and footers. It then calculates the correct reading order and can even extract complex mathematical formulas using LaTeX OCR.
With the release of Surya 2, the toolkit consolidated multiple heavy models into a single, highly optimized 650-million-parameter Vision Language Model (VLM). This means it is capable of processing over 5 pages per second on a modern GPU, yet remains small enough to run natively on an Apple Silicon MacBook using `llama.cpp`.
Who is it for?
- RAG Developers and Data Engineers who need to feed clean, structured text from visually complex PDFs and presentations into Large Language Models.
- Researchers and Academics digitizing historical documents or extracting data from dense academic papers containing nested tables and math formulas.
- Automation Teams looking to replace expensive, pay-per-page cloud OCR APIs with a free, locally hosted solution that guarantees data privacy.
- Archivists who need accurate reading order detection for multi-column documents, magazines, and newspapers.
What makes it special?
- 5-in-1 Pipeline — A single model handles full-page OCR, layout analysis, reading order, table recognition, and LaTeX extraction simultaneously.
- Unmatched Efficiency — Despite being a relatively tiny 650M parameter model, it scores an impressive 83.3% on the rigorous olmOCR benchmark, beating out massive 3B+ parameter models.
- Hardware Flexibility — It doesn’t lock you into expensive NVIDIA server setups. Surya ships with backends for both vLLM (for NVIDIA GPUs) and `llama.cpp` (for Apple Silicon and CPU inference).
- Seamless Integration — It serves as the foundational text-extraction engine for other popular open-source data parsing tools like Marker and Docling, making it the bedrock of modern document-to-markdown workflows.
Requirements before you start
To run Surya effectively, you will need a properly configured Python development environment:
- Python 3.10+ — Required to manage modern machine learning libraries.
- PyTorch — The core machine learning framework. You must install the correct version for your hardware (CUDA for NVIDIA, MPS for Apple Silicon).
- Sufficient Hardware — While it can run on a CPU, an NVIDIA GPU (RTX 3060 or better) or an Apple M-series chip is highly recommended for processing multi-page PDFs at reasonable speeds.
- pip — The standard Python package manager.
Step-by-step installation
Step 1 — Create a Virtual Environment
It is always best practice to isolate your Python dependencies so they don’t interfere with other AI tools on your system:
python -m venv surya_envActivate the environment:
- Windows:
surya_env\Scripts\activate - Mac/Linux:
source surya_env/bin/activate
Step 2 — Install PyTorch
Before installing Surya, ensure your system has the correct hardware acceleration libraries for PyTorch. Visit pytorch.org to get the exact command for your OS and GPU setup.
For example, to install with NVIDIA CUDA 12.1 support on Windows/Linux:
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121Step 3 — Install the Surya Toolkit
With your environment prepared, use pip to download the Surya OCR package. (Note: The model weights will automatically download the very first time you execute a command).
pip install surya-ocrStep 4 — Run Your First OCR Task
You can now use the command-line interface to extract text and bounding boxes from any document or image. Place a sample PDF (e.g., invoice.pdf) in your folder and run:
surya_ocr invoice.pdf --imagesSurya will process the document, generating a structured JSON file containing the extracted text, coordinate bounding boxes, and reading order. Because we added the --images flag, it will also save visual heatmaps showing exactly where it detected the text lines!
Common errors and fixes
| Error | What it means | How to fix it |
|---|---|---|
CUDA Out of Memory (OOM) | Your graphics card ran out of VRAM while trying to process a large PDF or high-resolution image. | You need to reduce the batch size. Set the environment variable RECOGNITION_BATCH_SIZE=8 (or lower) to limit how many pages the model tries to process at once. |
| Text lines are merged together, or faint text is ignored | The document layout is confusing the default detection thresholds. | You can manually tune the detection sensitivity by adjusting the DETECTOR_BLANK_THRESHOLD and DETECTOR_TEXT_THRESHOLD environment variables (between 0.0 and 1.0) to force the model to split or join text blocks. |
| Processing is incredibly slow | PyTorch has not detected your GPU and is falling back to your CPU. | Verify your PyTorch installation by running nvidia-smi (Windows/Linux) to ensure your CUDA drivers are active. If you are on an Apple M-chip, ensure you are utilizing the llama.cpp backend included in Surya 2. |
Free vs Paid comparison
| Feature | Surya OCR (Local Open Source) | Commercial OCR APIs (AWS Textract / Google Vision) |
|---|---|---|
| Cost | $0 (Free for personal/startup use) | $1.50+ per 1,000 pages |
| Data Privacy | ✅ Absolute — documents never leave your drive | ❌ Sent to third-party corporate servers |
| Table & Reading Order Layouts | ✅ Yes (Included natively) | Usually requires a more expensive API tier |
| Setup Required | ⚠️ High (Requires Python and GPU configuration) | 🟢 Low (Just an API key) |
Bottom line: If you are building automated RAG pipelines or processing thousands of highly sensitive PDF documents, Surya is a phenomenal, free alternative to paying Amazon or Google by the page. However, it requires technical know-how to host. If you have a massive enterprise budget and zero technical staff, a managed cloud service is easier to maintain.
Alternatives — 3 similar tools
1. Marker
Created by the exact same development team (Datalab), Marker actually uses Surya under the hood. However, while Surya outputs raw layout and coordinate JSON data, Marker is designed specifically to take a PDF and instantly convert it into perfectly formatted Markdown or HTML. If your end goal is Markdown, use Marker.
🔗 github.com/datalab-to/marker
2. Tesseract OCR
The grandfather of open-source OCR engines, maintained by Google. It is incredibly lightweight and fast, but it relies on older algorithmic methods. It struggles heavily with multi-column layouts, tables, and complex backgrounds compared to Surya’s modern AI vision models.
🔗 github.com/tesseract-ocr/tesseract
3. EasyOCR
A highly popular Python library that strikes a balance between ease of use and deep learning accuracy. It supports over 80 languages and is generally much easier to install on non-GPU machines than Surya, though it lacks Surya’s advanced table and LaTeX math recognition capabilities.
🚀 Want more free AI tools like this?
We find, test, and write setup guides for the best free and open-source AI tools — so you don’t have to dig through GitHub yourself.Browse Free AI Tools at globalaiforce.com/shop →
📸 Follow us for daily AI tool tips and tutorials: instagram.com/globalaiforce