Key Takeaways
- Apache 2.0 license, free and open source, no separate paid tier for the library itself
- Embedded by default — Faiss vector index plus SQLite metadata storage, both stored as local files
- One package covers vector search, RAG, agents, and multi-model workflows — not vector storage alone
- Built on Hugging Face Transformers, Sentence Transformers, and FastAPI; requires Python 3.10+
- Supports both local LLMs (Hugging Face, llama.cpp, Ollama, vLLM) and API-based models (OpenAI, Claude, AWS Bedrock via LiteLLM)
- Maintained by NeuML (creator David Mezzetti) — no VC-backed cloud product yet; a hosted txtai.cloud offering is still in development
📍 In One Sentence
txtai is a free, open-source (Apache 2.0) Python library that bundles a vector database, semantic search, RAG pipelines, and LLM orchestration into one embedded package with no separate server process.
💬 In Plain Terms
Instead of running Chroma or Qdrant as a background service and wiring a separate framework on top, you pip install txtai and get the vector store, the search, and the RAG plumbing inside your own Python program — the way SQLite lives inside an app instead of running as its own database server.
📌Note: txtai trades the horizontal scalability of a dedicated vector database service for zero deployment overhead. That trade makes sense for single-node applications and prototyping — not for datasets that need to shard across machines.
What Is txtai?
txtai is an open-source Python framework (Apache 2.0 license, github.com/neuml/txtai) for semantic search, LLM orchestration, and language model workflows, built and maintained by NeuML. Its core component is an embeddings database — described in its own documentation as a union of vector indexes (dense and sparse), graph networks, and relational databases in a single object.
- Vector search: dense and sparse embeddings, SQL filtering, topic modeling, graph analysis, and multimodal indexing (text, documents, audio, images, video) in one index
- Pipelines: pre-built wrappers around language models for question-answering, summarization, translation, transcription, and text labeling
- Workflows: chain multiple pipelines together into a single processing job, from a simple two-step script to a multi-model batch process
- Agents: autonomous agents that combine embeddings, pipelines, and workflows to work through multi-step tasks, built on the smolagents framework
- APIs and bindings: a REST/FastAPI service plus a Model Context Protocol (MCP) server, with client bindings for JavaScript, Java, Rust, and Go
- Over 70 example notebooks covering the framework end to end, maintained alongside the core library
How Does txtai's Embedded Architecture Work?
txtai's Embeddings object holds the vector index and metadata store directly in your Python process, persisting both to local files instead of talking to a separate database service. By default, the vector index uses Faiss and content metadata is stored in a local SQLite file — the same embed-in-the-app-process model SQLite itself uses, instead of a client/server model like PostgreSQL.
- ANN backend (
backendconfig): defaults to Faiss; HNSW, Annoy, and pgvector are supported as swappable alternatives without changing the rest of the code - Content storage (
contentconfig): defaults to SQLite when enabled; supports DuckDB or a client/server database via a connection URL for teams that outgrow a single file - Object storage: optional binary storage for images or arbitrary pickled objects, layered on top of the same embeddings index
- Persistence:
embeddings.save(path)writes the index and database to disk as a portable directory;embeddings.load(path)reopens it in a new process with no import/export step - No server process to start, monitor, or patch — the index lives and dies with your application process, the same as an in-memory or file-backed cache would
How Is txtai Different From a Standalone Vector Database?
Chroma, Qdrant, Weaviate, and Milvus normally run as their own service — a container or managed endpoint your application connects to over a network. txtai runs inside the calling process instead, the way SQLite differs from PostgreSQL: no connection string, no separate process to keep alive, no network hop between your code and the index.
What you gain by embedding the database
Zero infrastructure to provision — no Docker container, no managed cloud endpoint, no connection pooling. Deployment is copying a directory. For single-node applications, CLI tools, desktop apps, or prototypes, this removes an entire operational surface (uptime, auth, network latency) that a standalone vector database introduces even at small scale.
What you give up
A standalone vector database service can be queried by many independent processes at once, scaled horizontally across machines, and managed with dedicated ops tooling (backups, replicas, access control) separate from any one application's lifecycle. An embedded index is scoped to the process that opened it — concurrent writers from multiple processes are not the default use case, and very large collections that need to shard across nodes outgrow a single-file index.
📌Note: Chroma also supports an embedded mode for prototyping, but its production path is a server. txtai has no separate production mode to graduate into — embedded is the only architecture it offers.
Does txtai Support RAG and AI Agents?
Yes — retrieval-augmented generation and autonomous agents are core, first-class use cases in txtai, not add-ons bolted onto a vector store.
- RAG: the
RAGpipeline pairs anEmbeddingsindex with an LLM, retrieves relevant passages for a query, and generates an answer with citations back to source text — txtai's own example describes RAG as "more than vector search," also supporting context retrieval from web and SQL sources - Agents: built on the Hugging Face
smolagentsframework, txtai agents connect embeddings, pipelines, workflows, and other agents to work through multi-step tasks autonomously; agent prompting viaagents.mdandskill.mdfiles is supported - Workflows: pipelines chain together into linear or branching jobs — for example, extract text, chunk it, embed it, then summarize each chunk — without hand-rolling the glue code
- Knowledge graphs: LLM-driven entity extraction can build a semantic graph over an embeddings index, layering relationship analysis on top of plain similarity search
Which LLMs Can You Use With txtai?
txtai supports both local models and API-based models through the same LLM and RAG pipeline interfaces — switching between them is a configuration change, not a code rewrite.
Path | Type | Notes |
|---|---|---|
| Hugging Face Transformers | Local | Any causal LM on the Hugging Face Hub or a local path |
| llama.cpp | Local | GGUF-format quantized models, CPU or GPU |
| Ollama | Local | Points at a running Ollama server for model serving |
| vLLM | Local / self-hosted | High-throughput inference server for production |
| LiteLLM | API | Routes to OpenAI, Anthropic Claude, AWS Bedrock, and others |
The txtai RAG quickstart example loads a Hugging Face model by path string (for example Qwen/Qwen3-0.6B) directly into the RAG pipeline alongside the embeddings index — no separate LLM server is required unless you choose to run one for throughput reasons.
How Do You Set Up txtai?
Getting a working semantic search index running takes one pip install and a few lines of Python — there is no container to configure first.
- 1Install Python 3.10 or later, then install the package:
pip install txtai. Use `pip install "txtai[pipeline-data]"` if you also need document extraction (PDF, DOCX, HTML) for RAG. - 2Create an embeddings index in a Python script:
import txtaithenembeddings = txtai.Embeddings(). - 3Index a list of documents: `embeddings.index(["Correct", "Not what we hoped"])
. Each call adds text (or(id, text)` tuples for larger datasets) to the on-disk index. - 4Run a semantic search:
embeddings.search("positive", 1)returns the closest matches by meaning, not keyword overlap. - 5Persist the index for reuse:
embeddings.save("index_path")writes it to disk; reopen it later withembeddings.load("index_path")— no re-indexing needed between runs. - 6For a web API instead of an embedded script, define a minimal
app.ymlwith anembeddings.pathmodel, then serve it:CONFIG=app.yml uvicorn "txtai.api:app"and query it over HTTP withcurl.
import txtai
# Create an embeddings index (defaults to Faiss + local storage)
embeddings = txtai.Embeddings()
# Index text — each string becomes a searchable entry
embeddings.index(["Correct", "Not what we hoped"])
# Semantic search — finds meaning, not just keyword matches
results = embeddings.search("positive", 1)
print(results) # [(0, 0.298...)] — index 0 ("Correct") is the closest match
# Persist to disk for reuse across process restarts
embeddings.save("index_path")Does the minimal txtai example need a GPU?
No. The default embeddings model (sentence-transformers/all-MiniLM-L6-v2) and the Faiss ANN backend both run on CPU. A GPU speeds up embedding generation and LLM inference at larger scale but is not required to follow this setup.
How do I add retrieval-augmented generation to this setup?
Pass the same Embeddings object into a txtai.RAG pipeline alongside a local or API-based LLM: rag = txtai.RAG(embeddings, "model-name"), then call rag("your question"). The pipeline handles retrieval and prompt construction for you.
Who Should Use txtai?
Use txtai if you want one Python dependency for search, RAG, and agents with zero infrastructure to run — avoid it if you need a horizontally scalable vector store serving many independent applications.
Use txtai if
- You are building a single-node Python application, CLI tool, or desktop app and want vector search without deploying a database service
- You want RAG, agents, and multi-model workflows available from the same package instead of combining a vector database with a separate orchestration framework
- Your dataset fits comfortably on one machine — from a few thousand to low millions of vectors depending on dimension and available RAM
- You are prototyping and want to defer the "which vector database service do we deploy" decision until the project proves out
Avoid txtai if
- You need a vector store queried concurrently by many independent services or applications at once — that is a client/server database's job, not an embedded one
- Your collection needs to shard across multiple machines because it exceeds what a single node can hold in memory or on disk
- You want a visual, no-code workflow builder — txtai is a code-first Python library with no drag-and-drop canvas
- You need managed-service guarantees (SLA, dedicated ops tooling, access control separate from your application) that only a hosted vector database vendor provides today
📌Note: Final verdict: pick txtai when the constraint is "one Python app, one machine, minimal ops." Pick a standalone vector database (Qdrant, Weaviate, Milvus) when the constraint is "many services need to query the same index, at scale, from day one."
txtai vs. Chroma, Qdrant, and LlamaIndex
These four solve overlapping but distinct problems: txtai and Chroma both ship a vector store, Qdrant is a dedicated database service, and LlamaIndex is an orchestration framework with no built-in store of its own.
Tool | Architecture | Deployment | License | Best For |
|---|---|---|---|---|
| txtai | Embedded vector DB + RAG/agents | In-process, no server | Apache 2.0 | Single-package Python RAG & agents |
| Chroma | Vector database | Embedded or server mode | Apache 2.0 | Simple prototyping vector store |
| Qdrant | Vector database | Server (Docker/cloud) | Apache 2.0 | Scaled, multi-client production search |
| LlamaIndex | RAG/orchestration framework | Needs external vector store | MIT | Data connectors atop any vector DB |
Common Mistakes When Evaluating txtai
These mistakes come from applying assumptions about server-based vector databases to a library with a fundamentally different deployment model.
Mistake 1: Expecting txtai to scale like a hosted vector database out of the box
An embedded Faiss/SQLite index is bound to the process and machine that opened it. If the plan is many application instances querying one shared, growing index concurrently, that requires either a client/server content backend (via content: client with a connection URL) or a move to a dedicated vector database service — not the default embedded mode.
Mistake 2: Treating txtai as "just" a vector database
Evaluating txtai only on ANN search quality misses most of what the package does. Pipelines, workflows, agents, and knowledge graph construction are core modules, not bolt-ons — a fair comparison also weighs those against separately assembling a vector database plus LangChain, LlamaIndex, or a similar framework.
Mistake 3: Assuming NeuML sells a hosted version today
The open-source library carries no license fee. NeuML offers paid consulting around the txtai stack and, per its own documentation, is developing a separate hosted product called txtai.cloud — but as of this review that offering is still in progress, not a shipped, priced product to evaluate.
Frequently Asked Questions
Is txtai free to use?
Yes. txtai is open source under the Apache 2.0 license with no usage cap or license fee for the library itself. NeuML, the company that maintains it, sells paid AI consulting services and is separately developing a hosted product called txtai.cloud, still in development at the time of writing.
Does txtai require running a separate database server?
No. txtai embeds its vector index and metadata store directly inside your Python process — by default a Faiss ANN index plus a SQLite file, both persisted as local files with no server process to deploy or monitor.
What ANN backends does txtai support besides Faiss?
Faiss is the default. txtai also supports HNSW, Annoy, and pgvector (plus other backends via its ann extras package), configurable through the backend setting without changing application code.
How is txtai different from Chroma?
Both ship an embedded vector store, but Chroma's typical production path is running as a server, while txtai has no separate server mode to graduate into — it also bundles RAG pipelines, agents, and multi-model workflows in the same package, which Chroma does not.
How is txtai different from Qdrant?
Qdrant is a dedicated vector database service designed to run as its own process (via Docker or a managed cloud endpoint) and be queried by many clients at once. txtai runs embedded inside a single application process, trading that concurrency and horizontal scale for zero deployment overhead.
Does txtai support retrieval-augmented generation (RAG)?
Yes. The RAG pipeline combines an Embeddings index with a local or API-based LLM, retrieves relevant passages for a query, and generates a cited answer — txtai's own documentation frames RAG as more than vector search, also covering web and SQL context retrieval.
Can txtai use local LLMs instead of a cloud API?
Yes. txtai loads models through Hugging Face Transformers, llama.cpp (GGUF format), Ollama, or vLLM for fully local inference, or routes to OpenAI, Anthropic Claude, or AWS Bedrock via LiteLLM when an API-based model is preferred — the same LLM/RAG pipeline interface covers both.
Does txtai support AI agents?
Yes, built on the Hugging Face smolagents framework. txtai agents connect embeddings, pipelines, and workflows together to work through multi-step tasks autonomously, and support agent prompting conventions like agents.md and skill.md.
What license is txtai released under?
Apache License 2.0, which permits commercial use, modification, and redistribution without a royalty, the same permissive license used by Chroma and Qdrant.
Who maintains txtai?
txtai is developed and maintained by NeuML, a company founded by David Mezzetti. NeuML offers paid AI consulting services around the txtai stack alongside maintaining the open-source library.
Can txtai handle large datasets that do not fit on one machine?
Not in its default embedded mode. A single-file Faiss/SQLite index is scoped to the machine that holds it. Datasets that must shard across multiple nodes, or that need many independent services querying one shared index concurrently, are a better fit for a dedicated, horizontally scalable vector database.
Is txtai a good choice for a first RAG prototype?
Yes, for a Python developer specifically — the entire stack (index, RAG pipeline, and optionally an LLM) installs with one pip install txtai and runs in a single script, with no database container to stand up before writing the first line of application logic.