Key Takeaways
- Apache 2.0 license, free to self-host β pip install haystack-ai, source on github.com/deepset-ai/haystack
- Built by deepset, a Germany-based company, starting in 2019 as an extractive question-answering and search framework β before retrieval-augmented generation was a common term
- Two building blocks: Component (one processing step: retriever, embedder, generator, converter) and Pipeline (a connected graph of components with explicit .add_component() and .connect() calls)
- Document Store is a swappable backend abstraction β Haystack ships an in-memory store for testing and integrates with Elasticsearch, Weaviate, Pinecone, and other vector databases
- Model-provider agnostic: integrates with OpenAI, Anthropic, Mistral, Cohere, Hugging Face, Google, Azure OpenAI, and AWS Bedrock
- deepset also sells Haystack Enterprise Platform and Haystack Enterprise Starter β commercial layers with a visual pipeline builder and managed deployment on top of the same open-source core
π In One Sentence
Haystack is deepset's open-source (Apache 2.0) Python framework for building production search and RAG pipelines out of explicit, typed Component and Pipeline abstractions, currently at version 3.1.
π¬ In Plain Terms
Instead of chaining function calls like most LLM libraries, Haystack makes you wire named components β a retriever, a prompt builder, a generator β into a Pipeline object with explicit .connect() calls, so the data flow is visible and testable rather than hidden inside a chain.
πNote: The open-source framework and deepset's commercial products are separate: everything in this review is the free, self-hosted Apache 2.0 code unless a section explicitly says "Enterprise Platform."
What Is Haystack?
Haystack is an open-source Python framework (Apache 2.0, github.com/deepset-ai/haystack) for building search, question-answering, and retrieval-augmented generation (RAG) applications. It is maintained by deepset, a company based in Germany, and installed with pip install haystack-ai.
- Started in 2019 as an extractive question-answering framework β finding the exact answer span inside a document β before generative LLMs made RAG the dominant pattern
- Rewritten around Haystack 2.0 into the current Component/Pipeline architecture, now at version 3.1 (released 2026-08-24)
- Ships components for document conversion (PDF, HTML, DOCX), text splitting, embedding, retrieval (keyword-based BM25 and vector/semantic), generation, and evaluation
- Document Store abstraction decouples pipeline logic from the storage backend β swap the in-memory store for Elasticsearch, Weaviate, or Pinecone without rewriting the pipeline
- Ships an in-memory document store out of the box for local development and testing, with no external database required to get started
- Supports both keyword retrieval (BM25) and vector/semantic retrieval in the same pipeline, plus hybrid retrieval combining both
How Does Haystack's Pipeline/Component Architecture Work?
A Component is a single processing step (a retriever, an embedder, a generator) and a Pipeline is a directed graph of components connected with explicit .add_component() and .connect() calls, so the data flow between steps is visible in the code rather than hidden inside a chain object.
- Pipelines are serializable to YAML, so a pipeline built in Python can be saved, versioned, and reloaded without re-running the code that constructed it
- Branching and merging are native to the graph model β one retriever's output can feed two different generators, or two retrievers can feed one ranker, without extra glue code
- Because connections are explicit, a broken pipeline (mismatched types, a missing connection) fails at pipeline-build time with a clear error rather than at runtime deep inside a call stack
How Is Haystack Different From LangChain and LlamaIndex?
Haystack, LangChain, and LlamaIndex are all code-first Python frameworks with no visual builder in their open-source core β the difference is in their central abstraction. Haystack organizes everything around explicit Pipeline graphs of typed Components; LlamaIndex organizes around an Index built over your data with Retriever and QueryEngine objects sitting on top of it; LangChain organizes around chains and, in its LangGraph extension, a state graph for agents.
Does Haystack Support RAG and Production Search?
Yes β retrieval-augmented generation and production search are Haystack's core use cases, and the framework supports keyword search, vector search, and hybrid retrieval in the same pipeline.
- RAG pipelines: a retriever (BM25, vector, or hybrid) feeds relevant documents into a prompt builder, which feeds a generator (an LLM call) β the standard pattern this framework predates but now targets directly
- Retrievers: BM25 keyword retrieval and embedding-based (vector/semantic) retrieval are both first-class; hybrid retrieval combines the two and re-ranks the merged results
- Document Stores: an in-memory store ships for development; production deployments connect to Elasticsearch, Weaviate, Pinecone, or other supported vector databases through the same Component interface
- Agents: Haystack includes agent components that let a model decide which tool or pipeline step to call next, rather than following a fixed sequence
- Evaluation: Haystack ships evaluation components for measuring retrieval and generation quality against a labeled dataset, which matters for production search where relevance is measurable
How Do You Build a First Haystack Pipeline?
A minimal RAG pipeline needs three connected components: a retriever, a prompt builder, and a generator, wired into a Pipeline object and run with a query.
- 1Install the package: pip install haystack-ai.
- 2Import the pieces you need: from haystack import Pipeline, Document; from haystack.components.generators.chat import OpenAIChatGenerator; from haystack.components.retrievers import InMemoryBM25Retriever; from haystack.document_stores.in_memory import InMemoryDocumentStore; from haystack.components.builders import ChatPromptBuilder; from haystack.utils import Secret.
- 3Create a document store and write documents into it: document_store = InMemoryDocumentStore(), then document_store.write_documents([Document(content="...")β, ...]).
- 4Create the components: retriever = InMemoryBM25Retriever(document_store=document_store), prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*"), and llm = OpenAIChatGenerator(api_key=Secret.from_env_var("OPENAI_API_KEY"), model="gpt-4o-mini").
- 5Assemble the pipeline: rag_pipeline = Pipeline(), then rag_pipeline.add_component("retriever", retriever), rag_pipeline.add_component("prompt_builder", prompt_builder), and rag_pipeline.add_component("llm", llm).
- 6Wire the connections: rag_pipeline.connect("retriever", "prompt_builder.documents") and rag_pipeline.connect("prompt_builder", "llm").
- 7Run it: results = rag_pipeline.run({"retriever": {"query": question}, "prompt_builder": {"question": question}}).
from haystack import Pipeline, Document
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.retrievers import InMemoryBM25Retriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.builders import ChatPromptBuilder
from haystack.utils import Secret
document_store = InMemoryDocumentStore()
document_store.write_documents([
Document(content="My name is Jean and I live in Paris."),
Document(content="My name is Mark and I live in Berlin."),
Document(content="My name is Giorgio and I live in Rome."),
])
retriever = InMemoryBM25Retriever(document_store=document_store)
prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*")
llm = OpenAIChatGenerator(api_key=Secret.from_env_var("OPENAI_API_KEY"), model="gpt-4o-mini")
rag_pipeline = Pipeline()
rag_pipeline.add_component("retriever", retriever)
rag_pipeline.add_component("prompt_builder", prompt_builder)
rag_pipeline.add_component("llm", llm)
rag_pipeline.connect("retriever", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder", "llm")
results = rag_pipeline.run({"retriever": {"query": question}, "prompt_builder": {"question": question}})Do I need an OpenAI API key to try Haystack?
The example above uses OpenAIChatGenerator, but Haystack also has generator components for Hugging Face, Anthropic, Mistral, Cohere, Google, Azure OpenAI, and AWS Bedrock, plus local-model integrations β you are not locked into one model provider.
Do I need a vector database to get started?
No. InMemoryDocumentStore requires no external service and is enough to build and test a pipeline locally. Swap it for Elasticsearch, Weaviate, or Pinecone only when you move to production scale.
Who Should Use Haystack?
Haystack fits teams that need explicit control over a production search or RAG pipeline and are comfortable writing Python β it is not a no-code or visual tool at its open-source core.
Haystack vs. Alternatives
All three frameworks below are open-source, Python-first, and actively maintained β the choice comes down to which central abstraction matches how your team wants to reason about a pipeline.
Tool | Core Abstraction | License | Best For |
|---|---|---|---|
| Haystack | Pipeline of typed Components | Apache 2.0 | Production search / measurable RAG |
| LlamaIndex | Index + Retriever + QueryEngine | MIT | Fast data ingestion / indexing |
| LangChain | Chains / LangGraph state graph | MIT | General LLM app + agent glue code |
Common Mistakes When Evaluating Haystack
These mistakes come from treating Haystack as either a hosted SaaS product or a drop-in replacement for a different framework's abstraction.
Frequently Asked Questions
What is Haystack?
Haystack is an open-source Python framework by deepset, a Germany-based company, for building search, question-answering, and RAG pipelines. It is licensed Apache 2.0 and installed with pip install haystack-ai.
Who makes Haystack?
deepset, a company based in Germany. deepset started Haystack in 2019 as an extractive question-answering framework, before retrieval-augmented generation was a common term.
What license is Haystack released under?
Apache License 2.0. The source code is on github.com/deepset-ai/haystack and can be freely used, modified, and self-hosted.
What is the current version of Haystack?
Haystack 3.1, released August 24, 2026. The framework was substantially rewritten around Haystack 2.0 into its current Component/Pipeline architecture.
What is the difference between a Component and a Pipeline in Haystack?
A Component is one processing step β a retriever, embedder, prompt builder, or generator. A Pipeline is a graph of components connected with explicit .add_component() and .connect() calls; running the pipeline executes the graph in dependency order.
How is Haystack different from LlamaIndex?
Haystack organizes pipelines around explicit, typed Components wired into a Pipeline graph. LlamaIndex organizes around building an Index over your data and querying it through a Retriever and QueryEngine. LlamaIndex is typically faster to start indexing data with; Haystack's explicit graph gives more visibility and control for production pipelines.
How is Haystack different from LangChain?
Both are code-first Python frameworks with no visual builder in their open-source core. LangChain centers on chains and, via LangGraph, a state graph for agents. Haystack centers on a Pipeline of typed Components, with retrieval and search treated as first-class, measurable concerns rather than one link in a general-purpose chain.
Does Haystack require a specific vector database?
No. Haystack ships an in-memory Document Store for development and integrates with multiple vector databases and search backends, including Elasticsearch, Weaviate, and Pinecone, through the same Component interface β switching backends does not require rewriting the pipeline.
What is deepset's Haystack Enterprise Platform?
It is deepset's paid, commercial offering built on top of the open-source Haystack framework, adding a visual pipeline builder and managed or self-hosted deployment options. It is separate from the free, self-hosted, Apache 2.0 open-source framework this review covers.