Key Takeaways
- Apache 2.0 license, free to self-host — pip install haystack-ai, source on github.com/deepset-ai/haystack
- Built by deepset, a Germany-based company, starting in 2019 as an extractive question-answering and search framework — before retrieval-augmented generation was a common term
- Two building blocks: Component (one processing step: retriever, embedder, generator, converter) and Pipeline (a connected graph of components with explicit .add_component() and .connect() calls)
- Document Store is a swappable backend abstraction — Haystack ships an in-memory store for testing and integrates with Elasticsearch, Weaviate, Pinecone, and other vector databases
- Hayhooks, deepset's own tool, deploys a Haystack pipeline as a local REST API and MCP tool server, running on Uvicorn (default
http://localhost:1416) - Model-provider agnostic: integrates with OpenAI, Anthropic, Mistral, Cohere, Hugging Face, Google, Azure OpenAI, and AWS Bedrock
- deepset also sells Haystack Enterprise Platform and Haystack Enterprise Starter — commercial layers with a visual pipeline builder and managed deployment on top of the same open-source core
📍 In One Sentence
Haystack is deepset's open-source (Apache 2.0) Python framework for building production search and RAG pipelines out of explicit, typed Component and Pipeline abstractions, currently at version 3.2.0.
💬 In Plain Terms
Instead of chaining function calls like most LLM libraries, Haystack makes you wire named components — a retriever, a prompt builder, a generator — into a Pipeline object with explicit .connect() calls, so the data flow is visible and testable rather than hidden inside a chain.
📌Note: The open-source framework and deepset's commercial products are separate: everything in this review is the free, self-hosted Apache 2.0 code unless a section explicitly says "Enterprise Platform."
What Is Haystack?
Haystack is an open-source Python framework (Apache 2.0, github.com/deepset-ai/haystack) for building search, question-answering, and retrieval-augmented generation (RAG) applications. It is maintained by deepset, a company based in Germany, and installed with pip install haystack-ai.
- Started in 2019 as an extractive question-answering framework — finding the exact answer span inside a document — before generative LLMs made RAG the dominant pattern
- Rewritten around Haystack 2.0 into the current Component/Pipeline architecture, now at version 3.2.0 (released 2026-09-24)
- Ships components for document conversion (PDF, HTML, DOCX), text splitting, embedding, retrieval (keyword-based BM25 and vector/semantic), generation, and evaluation
- Document Store abstraction decouples pipeline logic from the storage backend — swap the in-memory store for Elasticsearch, Weaviate, or Pinecone without rewriting the pipeline
- Ships an in-memory document store out of the box for local development and testing, with no external database required to get started
- Supports both keyword retrieval (BM25) and vector/semantic retrieval in the same pipeline, plus hybrid retrieval combining both
How Does Haystack's Pipeline/Component Architecture Work?
A Component is a single processing step (a retriever, an embedder, a generator) and a Pipeline is a directed graph of components connected with explicit .add_component() and .connect() calls, so the data flow between steps is visible in the code rather than hidden inside a chain object.
- Pipelines are serializable to YAML, so a pipeline built in Python can be saved, versioned, and reloaded without re-running the code that constructed it
- Branching and merging are native to the graph model — one retriever's output can feed two different generators, or two retrievers can feed one ranker, without extra glue code
- Because connections are explicit, a broken pipeline (mismatched types, a missing connection) fails at pipeline-build time with a clear error rather than at runtime deep inside a call stack
Component
A Component wraps one unit of work — retrieve documents, embed text, build a prompt, call an LLM, convert a file. Each component declares typed inputs and outputs, so a component that outputs a list of Document objects can only connect to an input that accepts that type. Custom components can be written by subclassing the base Component class and defining run().
Pipeline
A Pipeline instance holds components and the connections between them. You call pipeline.add_component("name", component_instance) to register each step, then pipeline.connect("source.output", "target.input") to wire them together. Calling pipeline.run(...) executes the graph in dependency order and returns the outputs of every component that was not consumed downstream.
How Is Haystack Different From LangChain and LlamaIndex?
Haystack, LangChain, and LlamaIndex are all code-first Python frameworks with no visual builder in their open-source core — the difference is in their central abstraction. Haystack organizes everything around explicit Pipeline graphs of typed Components; LlamaIndex organizes around an Index built over your data with Retriever and QueryEngine objects sitting on top of it; LangChain organizes around chains and, in its LangGraph extension, a state graph for agents.
Haystack's Pipeline/Component model
Every step — retrieval, prompting, generation — is a named Component wired into a Pipeline with explicit .connect() calls. Nothing is implicit: if two components aren't connected, no data flows between them. This makes pipelines easy to inspect, unit-test one component at a time, and serialize to YAML for version control.
LlamaIndex's index-centric model
LlamaIndex centers on building an Index (vector, list, tree, or keyword) over a document collection, then querying it through a Retriever and QueryEngine. It is optimized for data ingestion and indexing convenience — a few lines can index a folder of files and start querying it — with less of Haystack's explicit step-by-step wiring.
What this means in practice
Haystack's explicit graph tends to pay off in production systems where you need to reason about exactly what runs, in what order, and swap one step (a different retriever, a different generator) without touching the rest. LlamaIndex's index abstraction tends to be faster to prototype with when the task is "index this data and let me query it."
Does Haystack Support RAG and Production Search?
Yes — retrieval-augmented generation and production search are Haystack's core use cases, and the framework supports keyword search, vector search, and hybrid retrieval in the same pipeline.
- RAG pipelines: a retriever (BM25, vector, or hybrid) feeds relevant documents into a prompt builder, which feeds a generator (an LLM call) — the standard pattern this framework predates but now targets directly
- Retrievers: BM25 keyword retrieval and embedding-based (vector/semantic) retrieval are both first-class; hybrid retrieval combines the two and re-ranks the merged results
- Document Stores: an in-memory store ships for development; production deployments connect to Elasticsearch, Weaviate, Pinecone, or other supported vector databases through the same Component interface
- Agents: Haystack includes agent components that let a model decide which tool or pipeline step to call next, rather than following a fixed sequence
- Evaluation: Haystack ships evaluation components for measuring retrieval and generation quality against a labeled dataset, which matters for production search where relevance is measurable
How Do You Build a First Haystack Pipeline?
A minimal RAG pipeline needs three connected components: a retriever, a prompt builder, and a generator, wired into a Pipeline object and run with a query.
- 1Install the package: pip install haystack-ai.
- 2Import the pieces you need: from haystack import Pipeline, Document; from haystack.components.generators.chat import OpenAIChatGenerator; from haystack.components.retrievers import InMemoryBM25Retriever; from haystack.document_stores.in_memory import InMemoryDocumentStore; from haystack.components.builders import ChatPromptBuilder; from haystack.utils import Secret.
- 3Create a document store and write documents into it: document_store = InMemoryDocumentStore(), then document_store.write_documents([Document(content="..."), ...]).
- 4Create the components: retriever = InMemoryBM25Retriever(document_store=document_store), prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*"), and llm = OpenAIChatGenerator(api_key=Secret.from_env_var("OPENAI_API_KEY"), model="gpt-4o-mini").
- 5Assemble the pipeline: rag_pipeline = Pipeline(), then rag_pipeline.add_component("retriever", retriever), rag_pipeline.add_component("prompt_builder", prompt_builder), and rag_pipeline.add_component("llm", llm).
- 6Wire the connections: rag_pipeline.connect("retriever", "prompt_builder.documents") and rag_pipeline.connect("prompt_builder", "llm").
- 7Run it: results = rag_pipeline.run({"retriever": {"query": question}, "prompt_builder": {"question": question}}).
from haystack import Pipeline, Document
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.retrievers import InMemoryBM25Retriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.builders import ChatPromptBuilder
from haystack.utils import Secret
document_store = InMemoryDocumentStore()
document_store.write_documents([
Document(content="My name is Jean and I live in Paris."),
Document(content="My name is Mark and I live in Berlin."),
Document(content="My name is Giorgio and I live in Rome."),
])
retriever = InMemoryBM25Retriever(document_store=document_store)
prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*")
llm = OpenAIChatGenerator(api_key=Secret.from_env_var("OPENAI_API_KEY"), model="gpt-4o-mini")
rag_pipeline = Pipeline()
rag_pipeline.add_component("retriever", retriever)
rag_pipeline.add_component("prompt_builder", prompt_builder)
rag_pipeline.add_component("llm", llm)
rag_pipeline.connect("retriever", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder", "llm")
results = rag_pipeline.run({"retriever": {"query": question}, "prompt_builder": {"question": question}})Do I need an OpenAI API key to try Haystack?
The example above uses OpenAIChatGenerator, but Haystack also has generator components for Hugging Face, Anthropic, Mistral, Cohere, Google, Azure OpenAI, and AWS Bedrock, plus local-model integrations — you are not locked into one model provider.
Do I need a vector database to get started?
No. InMemoryDocumentStore requires no external service and is enough to build and test a pipeline locally. Swap it for Elasticsearch, Weaviate, or Pinecone only when you move to production scale.
Who Should Use Haystack?
Haystack fits teams that need explicit control over a production search or RAG pipeline and are comfortable writing Python — it is not a no-code or visual tool at its open-source core.
Use Haystack if
- You are building a production search, question-answering, or RAG system where retrieval quality needs to be measured and tuned, not just plugged in
- You want an explicit, inspectable pipeline graph you can unit-test component by component, rather than an implicit chain
- You need to swap retrievers, document stores, or model providers without rewriting the rest of the pipeline
- You are comfortable in Python and want a framework, not a hosted product — Haystack is self-hosted, code-first, and free under Apache 2.0
Avoid Haystack if
- You want a no-code or visual builder as your primary interface — the open-source framework is code-first; a visual layer exists only in deepset's paid Enterprise Platform
- Your task is closer to "index this folder of files and let me query it" with minimal setup — LlamaIndex's index-centric API is typically faster to get to a first answer for that specific shape of task
- You need a JavaScript/TypeScript-native framework — Haystack's open-source SDK is Python-only
- You have no interest in search-quality metrics (BM25 vs. vector vs. hybrid, retrieval evaluation) and just want an LLM to answer questions over a handful of documents with no tuning
Haystack vs. Alternatives
All three frameworks below are open-source, Python-first, and actively maintained — the choice comes down to which central abstraction matches how your team wants to reason about a pipeline.
Tool | Core Abstraction | License | Best For |
|---|---|---|---|
| Haystack | Pipeline of typed Components | Apache 2.0 | Production search / measurable RAG |
| LlamaIndex | Index + Retriever + QueryEngine | MIT | Fast data ingestion / indexing |
| LangChain | Chains / LangGraph state graph | MIT | General LLM app + agent glue code |
Common Mistakes When Evaluating Haystack
These mistakes come from treating Haystack as either a hosted SaaS product or a drop-in replacement for a different framework's abstraction.
Mistake 1: Assuming Haystack is a hosted product
The open-source framework (pip install haystack-ai) is self-hosted code, not a SaaS you sign up for. deepset's commercial offerings — Haystack Enterprise Platform and Haystack Enterprise Starter — are separate paid layers on top of the same open-source core, not a requirement to use it.
Mistake 2: Expecting LlamaIndex-style "index a folder in three lines" convenience
Haystack's explicit Pipeline/Component model trades some of that quick-start convenience for visibility and control. If your task is purely "point at a folder, start querying," expect more setup than LlamaIndex's index-first API — the payoff shows up later, when you need to swap or debug one step in production.
Mistake 3: Ignoring the Document Store choice until production
InMemoryDocumentStore is convenient for development but is not meant for production scale or persistence. Decide on the production Document Store (Elasticsearch, Weaviate, Pinecone, or another supported backend) early, since retriever behavior and indexing strategy can differ by backend.
Frequently Asked Questions
What is Haystack?
Haystack is an open-source Python framework by deepset, a Germany-based company, for building search, question-answering, and RAG pipelines. It is licensed Apache 2.0 and installed with pip install haystack-ai.
Who makes Haystack?
deepset, a company based in Germany. deepset started Haystack in 2019 as an extractive question-answering framework, before retrieval-augmented generation was a common term.
What license is Haystack released under?
Apache License 2.0. The source code is on github.com/deepset-ai/haystack and can be freely used, modified, and self-hosted.
What is the current version of Haystack?
Haystack 3.2.0, released September 24, 2026. The framework was substantially rewritten around Haystack 2.0 into its current Component/Pipeline architecture.
What is the difference between a Component and a Pipeline in Haystack?
A Component is one processing step — a retriever, embedder, prompt builder, or generator. A Pipeline is a graph of components connected with explicit .add_component() and .connect() calls; running the pipeline executes the graph in dependency order.
How is Haystack different from LlamaIndex?
Haystack organizes pipelines around explicit, typed Components wired into a Pipeline graph. LlamaIndex organizes around building an Index over your data and querying it through a Retriever and QueryEngine. LlamaIndex is typically faster to start indexing data with; Haystack's explicit graph gives more visibility and control for production pipelines.
How is Haystack different from LangChain?
Both are code-first Python frameworks with no visual builder in their open-source core. LangChain centers on chains and, via LangGraph, a state graph for agents. Haystack centers on a Pipeline of typed Components, with retrieval and search treated as first-class, measurable concerns rather than one link in a general-purpose chain.
Does Haystack require a specific vector database?
No. Haystack ships an in-memory Document Store for development and integrates with multiple vector databases and search backends, including Elasticsearch, Weaviate, and Pinecone, through the same Component interface — switching backends does not require rewriting the pipeline.
What is deepset's Haystack Enterprise Platform?
It is deepset's paid, commercial offering built on top of the open-source Haystack framework, adding a visual pipeline builder and managed or self-hosted deployment options. It is separate from the free, self-hosted, Apache 2.0 open-source framework this review covers.