🟢 Beginner–Intermediate ⚙️ Type: Headless Browser / AI Scraping 💸 Free & Open Source (Apache-2.0) ⭐ 15,600+ GitHub Stars
What is Obscura?
Obscura is an open-source, ultra-lightweight headless browser engine written entirely in Rust. It was built from the ground up to solve the glaring issues with traditional headless browsers (like Chrome or Firefox) when used for AI agents and web scraping: extreme memory bloat, slow startup times, and instant detection by anti-bot systems.
Instead of forcing an AI agent to launch a 300+ MB instance of Chromium just to read a webpage, Obscura provides the same functionality in a fraction of the footprint. It executes real JavaScript via the V8 engine and speaks the standard Chrome DevTools Protocol (CDP). This means it acts as a flawless, drop-in replacement for headless Chrome in popular automation frameworks like Puppeteer and Playwright.
Beyond being a standard scraper, Obscura natively implements features specifically for Large Language Models (LLMs), including an integrated Model Context Protocol (MCP) server and a built-in function to instantly convert cluttered website DOMs into clean, token-efficient Markdown.
Who is it for?
- AI Agent Developers building autonomous systems (like Claude Code or custom MCP agents) that need to browse the web, read documentation, or interact with web apps without devouring gigabytes of RAM.
- Data Engineers and Web Scrapers tired of getting blocked by Cloudflare or generic anti-bot protections when trying to extract public data.
- Automation Testers looking to drastically speed up their CI/CD pipelines by replacing heavy Chrome instances with a browser that starts up in under 100 milliseconds.
- Homelabbers running local AI workflows on constrained hardware who cannot afford the massive overhead of standard browser automation.
What makes it special?
- Blazing Fast and Micro-Sized — The binary is roughly 70 MB and uses around 30 MB of runtime memory. It boots up in a staggering 85 milliseconds, compared to the ~500ms startup time of traditional headless Chrome.
- Built-In Stealth Mode — Turn on the
--stealthflag and Obscura automatically randomizes session fingerprints, modifiesnavigator.webdriver, and blocks over 3,500 known tracker and telemetry domains natively, making it incredibly difficult for sites to detect it as a bot. - Zero Dependencies — It is a standalone, compiled Rust binary. You do not need to install Chrome, Node.js, Python, or complex web drivers to run it from the terminal.
- AI-Ready Markdown Extraction — It features a custom CDP method designed purely for LLMs. It strips out headers, footers, and noise, outputting perfectly formatted, token-friendly Markdown of the article content.
- Parallel Scraping Command — It includes a built-in
scrapesubcommand that allows you to fetch multiple URLs concurrently across multiple workers straight from the CLI.
Requirements before you start
Obscura is incredibly forgiving on system resources, but you will need the following depending on how you plan to use it:
- A Modern Operating System — Pre-built binaries are available for Windows, macOS (Intel and Apple Silicon), and Linux (x86_64 and AArch64).
- (Optional) Puppeteer or Playwright — If you want to drive Obscura programmatically using JavaScript/Python, you will need your preferred automation library installed.
- (Optional) Rust 1.75+ — Only required if you intend to compile the browser engine from the source code yourself.
Step-by-step installation
Method 1 — Download Pre-Built Binaries (Easiest)
If you just want to run Obscura instantly without compiling code, follow these steps:
- Navigate to the Obscura GitHub Releases page.
- Download the archive that matches your operating system (e.g.,
obscura-x86_64-windows.ziporobscura-aarch64-macos.tar.gz). - Extract the folder. You will see two core files:
obscuraandobscura-worker. These must remain in the same directory. - (Optional but recommended) Add the folder path to your system’s Environment Variables (PATH) so you can type
obscurafrom any terminal window.
Method 2 — Build from Source
If you prefer to compile it yourself for maximum performance or custom flags:
git clone https://github.com/h4ckf0r0day/obscura.git
cd obscura
cargo build --release(Note: The first build will take around 5 minutes as it compiles the V8 JavaScript engine from scratch).
Step 3 — Run Your First AI Fetch
Open your terminal and test the engine by fetching a webpage and instantly converting it to AI-friendly Markdown:
obscura fetch https://example.com --stealth --dump markdownStep 4 — Connect Puppeteer or Playwright
To use Obscura as a drop-in replacement for Chrome in your existing automation scripts, start the background server first:
obscura serve --stealth --port 9222Then, modify your Node.js script to connect to that WebSocket endpoint rather than launching a new browser:
const browser = await puppeteer.connect({
browserWSEndpoint: 'ws://localhost:9222/devtools/browser/'
});Common errors and fixes
| Error | What it means | How to fix it |
|---|---|---|
Connection Refused (Puppeteer/Playwright) | Your automation script is trying to connect to Obscura, but the background CDP server isn’t running. | You must open a terminal and execute obscura serve --port 9222 before you execute your JavaScript/Python scraping code. |
Missing obscura-worker executable | You moved the main obscura binary to a different folder and separated it from its required worker executable. | Obscura requires both binaries to be in the exact same directory to handle multi-threaded scraping. Move them back together. |
| App flagged as malware by AI or Antivirus | Some AI tools (like Claude) and strict corporate firewalls occasionally flag the author’s handle (h4ckf0r0day) as suspicious. | This is a false positive based purely on the username string. Download and install the binary manually rather than asking an AI agent to do it for you. |
Free vs Paid comparison
| Feature | Obscura (Local Open Source) | Browserless / Steel.dev (Cloud APIs) |
|---|---|---|
| Cost | $0 (Free Apache-2.0 License) | Monthly subscription or pay-per-request |
| Memory Footprint | 🟢 ~30 MB Runtime | Offloaded to the cloud provider |
| Residential Proxy Network | ❌ No (You must route it through your own proxies) | ✅ Included in managed services |
| Setup Required | ⚠️ Requires manual execution and server management | 🟢 Instant API endpoints |
Bottom line: Obscura is an absolute masterclass in software optimization. If you are running multiple AI agents that need to browse the web, doing it through standard headless Chrome will obliterate your computer’s RAM. Obscura solves this elegantly and natively supports the Markdown formatting that LLMs crave. If you don’t want to pay a monthly fee for a cloud browser API, this is the ultimate local tool.
Alternatives — 3 similar tools
1. Crawl4AI
An incredibly popular Python-based framework designed specifically for LLM web crawling. Like Obscura, it specializes in returning clean Markdown and JSON data, but it relies on Playwright and standard headless browsers under the hood, making it significantly heavier and slower to start than Obscura’s native Rust implementation.
🔗 github.com/unclecode/crawl4ai
2. Steel.dev
An open-source (and commercially hosted) headless browser API explicitly built for AI agents. It handles all the heavy lifting of managing browser instances, session persistence, and anti-bot bypassing. It is a fantastic option if you prefer to interact with browsers via a remote API rather than running a local engine.
🔗 github.com/steel-dev/steel-browser
3. Jina Reader API
If you don’t want to install any browser software at all, Jina Reader is a brilliant, free-tier hosted service. You simply prepend r.jina.ai/ to any URL, and their cloud servers will instantly scrape the page, render the JavaScript, and return an LLM-friendly Markdown string directly to your application.
🚀 Want more free AI tools like this?
We find, test, and write setup guides for the best free and open-source AI tools — so you don’t have to dig through GitHub yourself.Browse Free AI Tools at globalaiforce.com/shop →
📸 Follow us for daily AI tool tips and tutorials: instagram.com/globalaiforce