Skip to main content
PromptQuorumBuilt for humans. Structured for AI.
Home/Power Local LLM/Firecrawl Review: Open-Source Web Data API for LLM Pipelines
Local AI Agents & Tool Use

Firecrawl Review: Open-Source Web Data API for LLM Pipelines

·8 min read·By Hans Kuepper · Founder of PromptQuorum · Discovery engine for open-weight & open-source AI
Read in:
🇺🇸en🇩🇪de🇫🇷fr🇯🇵ja🇨🇳zh🇪🇸es🇧🇷pt🇸🇦ar🇰🇷ko

Firecrawl is an open-source (AGPL-3.0) web data API that searches, scrapes, crawls and maps websites and returns clean markdown or JSON for LLM pipelines. It runs as a hosted service or on your own servers through Docker Compose, and the self-hosted edition is a reduced version of the cloud product.

Firecrawl (firecrawl.dev, source at github.com/firecrawl/firecrawl) is a web data API that turns websites into markdown, HTML, screenshots or structured JSON for LLM applications. It belongs in a local-AI directory because its AGPL-3.0 core can be self-hosted with Docker Compose and its AI-backed features can talk to Ollama instead of a cloud model. This review covers what Firecrawl does, what the self-hosted stack leaves out, how the hosted pricing works, and who it fits.

Key Takeaways

  • The hosted API has a free tier and paid plans from $16 per month billed yearly; self-hosting costs no licence fee but you run the servers
  • Self-hosting omits the proprietary Fire-engine, screenshots, page actions, Agent and interact features
  • Local-model extraction through Ollama is supported, but Firecrawl's own docs tell you to test that path separately
  • The default self-hosted API is unauthenticated, so it must stay on a trusted network until you add your own authentication
  • Firecrawl's privacy policy states that its servers are in the United States

📍 In One Sentence

Firecrawl converts websites into LLM-ready markdown or JSON and is offered as a hosted service with a free tier or as source you can self-host.

💬 In Plain Terms

Firecrawl visits web pages for you and hands back clean text that a language model can read. You can pay for the hosted version or run a smaller copy on your own server. If you point that copy at a local Ollama model, the extraction step stays on your hardware too.

What Is Firecrawl?

Firecrawl is a web data API for AI systems: you send it a URL or a search query, and it returns the page content in a format an LLM can use directly. Its repository describes it as a way to "supercharge your AI agents with data from the web," and the company runs the hosted service at firecrawl.dev.

  • Product type: a REST API with SDKs and a CLI, offered as a hosted service and as source code you can self-host
  • Repository: github.com/firecrawl/firecrawl, 192,265 stars, verified via the GitHub API
  • Licence: AGPL-3.0 for the open-source repository, per its LICENSE file. The Fire-engine proxy and rendering layer used by the hosted service is proprietary and not in the repository
  • Hosted-service compliance: the site shows a SOC 2 Type II badge, which this review has not independently audited

What Can You Do With Firecrawl?

Firecrawl groups its endpoints by how much of the web you want to read at once. The first four below are available in the self-hosted default stack; the last two are cloud features.

  • Scrape — fetch one URL and return markdown, HTML, metadata or schema-based JSON; PDFs and DOCX files hosted on the web are parsed too
  • Crawl — scrape all reachable pages of a site from a single request
  • Map — list the URLs of a site without scraping their content
  • Search — run a web search and return results with full-page content
  • Interact — scrape a page, then click, navigate and operate it with prompts or code (cloud feature)
  • Agent — describe the data you need and let Firecrawl gather it autonomously (cloud feature)

Self-Hosted vs. Hosted Firecrawl

Self-hosted Firecrawl is a Docker Compose stack that covers the core scrape, crawl, map and search routes; the hosted service adds Fire-engine and several features on top. The repository's SELF_HOST.md and the self-hosting guide are explicit about the gap.

  • Stack: the Compose file runs the API and workers, Playwright, Redis, RabbitMQ, a PostgreSQL queue and an optional FoundationDB backend; only the API is published, on port 3002
  • Requirements: Git, Docker with Compose v2 and curl; Firecrawl publishes no verified minimum host size
  • Before production: the default API has no authentication and the Compose file defines no persistent volumes, so add authentication, TLS, backups and monitoring first

Core routes

Self-hosted stack:
Scrape, crawl, map, search
Hosted service:
All, plus extras

Anti-bot engine

Self-hosted stack:
Not included; run Fire-engine separately
Hosted service:
Fire-engine (proprietary)

Screenshots and actions

Self-hosted stack:
Not in the default stack
Hosted service:
Yes

Agent and interact

Self-hosted stack:
Use Firecrawl Cloud instead
Hosted service:
Yes

LLM-backed formats

Self-hosted stack:
Connect Ollama or an OpenAI-compatible provider
Hosted service:
Managed by Firecrawl

Operations

Self-hosted stack:
You own security, uptime and upgrades
Hosted service:
Firecrawl operates it

Because the licence is AGPL-3.0, check its network-use terms with your legal team before offering a modified copy to others as a service.

Firecrawl Pricing

Firecrawl's hosted plans are priced in credits, starting with a free tier of 1,000 credits per month. Prices below come from firecrawl.dev/pricing and are listed in US dollars only.

  • Scrape, crawl and map cost 1 credit per page; search costs 2 credits per 10 results; interact costs 2 credits per browser minute
  • The JSON, question and highlight formats add 4 credits per page on scrape and crawl
  • A scrape that returns no result is not charged; an error page such as 403 or 404 still costs 1 credit
  • Credits roll over only on Scale and Enterprise; payments run through Stripe and a monthly spend cap can be set

Free

Price:
$0
Credits and concurrency:
1,000 credits, 2 concurrent

Hobby

Price:
$16/month yearly ($19 monthly)
Credits and concurrency:
5,000 credits, 5 concurrent

Standard

Price:
$83/month billed yearly
Credits and concurrency:
100,000 credits, 25 concurrent

Growth

Price:
$333/month billed yearly
Credits and concurrency:
500,000 credits, 50 concurrent

Scale

Price:
$599/month billed yearly
Credits and concurrency:
1,000,000 credits, 100 concurrent

Enterprise

Price:
Custom
Credits and concurrency:
Custom

The pricing page lists a monthly price for Hobby only and its stated yearly saving does not match the $3 monthly difference, so confirm current figures at checkout.

Who Should Use Firecrawl?

Firecrawl fits developers who need web content in an LLM-ready format and would rather not build their own crawler and renderer.

Use Firecrawl if you

  • Feed web pages into RAG, agents or fine-tuning pipelines and want markdown or JSON instead of raw HTML
  • Connect an MCP-capable client such as Claude, Cursor or Windsurf to live web data
  • Can operate a Docker Compose stack and want the core routes on infrastructure you control

Look elsewhere if you

  • Need screenshots, page actions or Agent without paying for the hosted service
  • Want a double-click desktop app with no terminal or API key
  • Cannot accept a US-based processor for scraped content on the hosted plan

Competitors and Alternatives

The directory lists no other scraping or crawling service, so the nearest same-category reviews are frameworks that consume web data rather than fetch it. Firecrawl sits upstream of them: it produces the clean text, and a framework decides what to do with it.

Pydantic AI

Best known for:
Type-safe agent framework built on Pydantic
Link:
Pydantic AI Review
Articles about Pydantic AI (2)

DSPy

Best known for:
Programming, not prompting, language-model pipelines
Link:
DSPy Review
Articles about DSPy (1)

Also mentioned in:

See the Local LLM Software Directory for the full catalog, including Firecrawl's own entry.

Compare in directory

Frequently Asked Questions

Is Firecrawl open source?

The repository is licensed AGPL-3.0, but the Fire-engine layer used by the hosted service is proprietary and not part of it.

Does Firecrawl work with Ollama?

Yes for LLM-backed features in a self-hosted stack, per Firecrawl's self-hosting guide, which also advises testing that path separately. This review did not run it.

Is there an official MCP server?

Yes. Firecrawl publishes firecrawl-mcp-server under the MIT licence for clients such as Claude, Cursor and Windsurf.

How much RAM does self-hosting need?

Firecrawl states that it does not publish a verified minimum host size; the stack runs several services, so plan capacity and test it.

Where is scraped data stored on the hosted plan?

The privacy policy says its servers are in the United States and that data may pass through other countries in transit. It does not mention model training.

Has PromptQuorum tested Firecrawl?

No. This review is based on the official site, documentation, GitHub repository and pricing page, not on hands-on benchmarks.

Sources

← Back to Power Local LLM

MCP