Key Takeaways
- Apache 2.0 license, free and open source, no separate paid tier for the library itself
- Embedded by default — Faiss vector index plus SQLite metadata storage, both stored as local files
- One package covers vector search, RAG, agents, and multi-model workflows — not vector storage alone
- Built on Hugging Face Transformers, Sentence Transformers, and FastAPI; requires Python 3.10+
- Supports both local LLMs (Hugging Face, llama.cpp, Ollama, vLLM) and API-based models (OpenAI, Claude, AWS Bedrock via LiteLLM)
- Maintained by NeuML (creator David Mezzetti) — no VC-backed cloud product yet; a hosted txtai.cloud offering is still in development
📍 In One Sentence
txtai is a free, open-source (Apache 2.0) Python library that bundles a vector database, semantic search, RAG pipelines, and LLM orchestration into one embedded package with no separate server process.
💬 In Plain Terms
Instead of running Chroma or Qdrant as a background service and wiring a separate framework on top, you pip install txtai and get the vector store, the search, and the RAG plumbing inside your own Python program — the way SQLite lives inside an app instead of running as its own database server.
📌Note: txtai trades the horizontal scalability of a dedicated vector database service for zero deployment overhead. That trade makes sense for single-node applications and prototyping — not for datasets that need to shard across machines.
What Is txtai?
txtai is an open-source Python framework (Apache 2.0 license, github.com/neuml/txtai) for semantic search, LLM orchestration, and language model workflows, built and maintained by NeuML. Its core component is an embeddings database — described in its own documentation as a union of vector indexes (dense and sparse), graph networks, and relational databases in a single object.
- Vector search: dense and sparse embeddings, SQL filtering, topic modeling, graph analysis, and multimodal indexing (text, documents, audio, images, video) in one index
- Pipelines: pre-built wrappers around language models for question-answering, summarization, translation, transcription, and text labeling
- Workflows: chain multiple pipelines together into a single processing job, from a simple two-step script to a multi-model batch process
- Agents: autonomous agents that combine embeddings, pipelines, and workflows to work through multi-step tasks, built on the smolagents framework
- APIs and bindings: a REST/FastAPI service plus a Model Context Protocol (MCP) server, with client bindings for JavaScript, Java, Rust, and Go
- Over 70 example notebooks covering the framework end to end, maintained alongside the core library
How Does txtai's Embedded Architecture Work?
**txtai's Embeddings object holds the vector index and metadata store directly in your Python process, persisting both to local files instead of talking to a separate database service.** By default, the vector index uses Faiss and content metadata is stored in a local SQLite file — the same embed-in-the-app-process model SQLite itself uses, instead of a client/server model like PostgreSQL.
- ANN backend (
backendconfig): defaults to Faiss; HNSW, Annoy, and pgvector are supported as swappable alternatives without changing the rest of the code - Content storage (
contentconfig): defaults to SQLite when enabled; supports DuckDB or a client/server database via a connection URL for teams that outgrow a single file - Object storage: optional binary storage for images or arbitrary pickled objects, layered on top of the same embeddings index
- Persistence:
embeddings.save(path)writes the index and database to disk as a portable directory;embeddings.load(path)reopens it in a new process with no import/export step - No server process to start, monitor, or patch — the index lives and dies with your application process, the same as an in-memory or file-backed cache would
How Is txtai Different From a Standalone Vector Database?
Chroma, Qdrant, Weaviate, and Milvus normally run as their own service — a container or managed endpoint your application connects to over a network. txtai runs inside the calling process instead, the way SQLite differs from PostgreSQL: no connection string, no separate process to keep alive, no network hop between your code and the index.
📌Note: Chroma also supports an embedded mode for prototyping, but its production path is a server. txtai has no separate production mode to graduate into — embedded is the only architecture it offers.
Does txtai Support RAG and AI Agents?
Yes — retrieval-augmented generation and autonomous agents are core, first-class use cases in txtai, not add-ons bolted onto a vector store.
- RAG: the
RAGpipeline pairs anEmbeddingsindex with an LLM, retrieves relevant passages for a query, and generates an answer with citations back to source text — txtai's own example describes RAG as "more than vector search," also supporting context retrieval from web and SQL sources - Agents: built on the Hugging Face
smolagentsframework, txtai agents connect embeddings, pipelines, workflows, and other agents to work through multi-step tasks autonomously; agent prompting viaagents.mdandskill.mdfiles is supported - Workflows: pipelines chain together into linear or branching jobs — for example, extract text, chunk it, embed it, then summarize each chunk — without hand-rolling the glue code
- Knowledge graphs: LLM-driven entity extraction can build a semantic graph over an embeddings index, layering relationship analysis on top of plain similarity search
Which LLMs Can You Use With txtai?
**txtai supports both local models and API-based models through the same LLM and RAG pipeline interfaces — switching between them is a configuration change, not a code rewrite.**
Path | Type | Notes |
|---|---|---|
| Hugging Face Transformers | Local | Any causal LM on the Hugging Face Hub or a local path |
| llama.cpp | Local | GGUF-format quantized models, CPU or GPU |
| Ollama | Local | Points at a running Ollama server for model serving |
| vLLM | Local / self-hosted | High-throughput inference server for production |
| LiteLLM | API | Routes to OpenAI, Anthropic Claude, AWS Bedrock, and others |
The txtai RAG quickstart example loads a Hugging Face model by path string (for example Qwen/Qwen3-0.6B) directly into the RAG pipeline alongside the embeddings index — no separate LLM server is required unless you choose to run one for throughput reasons.
How Do You Set Up txtai?
Getting a working semantic search index running takes one pip install and a few lines of Python — there is no container to configure first.
- 1Install Python 3.10 or later, then install the package:
pip install txtai. Use `pip install "txtai[pipeline-data]"` if you also need document extraction (PDF, DOCX, HTML) for RAG. - 2Create an embeddings index in a Python script:
import txtaithenembeddings = txtai.Embeddings(). - 3Index a list of documents: `embeddings.index(["Correct", "Not what we hoped"])
. Each call adds text (or(id, text)` tuples for larger datasets) to the on-disk index. - 4Run a semantic search:
embeddings.search("positive", 1)returns the closest matches by meaning, not keyword overlap. - 5Persist the index for reuse:
embeddings.save("index_path")writes it to disk; reopen it later withembeddings.load("index_path")— no re-indexing needed between runs. - 6For a web API instead of an embedded script, define a minimal
app.ymlwith anembeddings.pathmodel, then serve it:CONFIG=app.yml uvicorn "txtai.api:app"and query it over HTTP withcurl.
import txtai
# Create an embeddings index (defaults to Faiss + local storage)
embeddings = txtai.Embeddings()
# Index text — each string becomes a searchable entry
embeddings.index(["Correct", "Not what we hoped"])
# Semantic search — finds meaning, not just keyword matches
results = embeddings.search("positive", 1)
print(results) # [(0, 0.298...)] — index 0 ("Correct") is the closest match
# Persist to disk for reuse across process restarts
embeddings.save("index_path")Does the minimal txtai example need a GPU?
No. The default embeddings model (sentence-transformers/all-MiniLM-L6-v2) and the Faiss ANN backend both run on CPU. A GPU speeds up embedding generation and LLM inference at larger scale but is not required to follow this setup.
How do I add retrieval-augmented generation to this setup?
Pass the same Embeddings object into a txtai.RAG pipeline alongside a local or API-based LLM: rag = txtai.RAG(embeddings, "model-name"), then call rag("your question"). The pipeline handles retrieval and prompt construction for you.
Who Should Use txtai?
Use txtai if you want one Python dependency for search, RAG, and agents with zero infrastructure to run — avoid it if you need a horizontally scalable vector store serving many independent applications.
📌Note: Final verdict: pick txtai when the constraint is "one Python app, one machine, minimal ops." Pick a standalone vector database (Qdrant, Weaviate, Milvus) when the constraint is "many services need to query the same index, at scale, from day one."
txtai vs. Chroma, Qdrant, and LlamaIndex
These four solve overlapping but distinct problems: txtai and Chroma both ship a vector store, Qdrant is a dedicated database service, and LlamaIndex is an orchestration framework with no built-in store of its own.
Tool | Architecture | Deployment | License | Best For |
|---|---|---|---|---|
| txtai | Embedded vector DB + RAG/agents | In-process, no server | Apache 2.0 | Single-package Python RAG & agents |
| Chroma | Vector database | Embedded or server mode | Apache 2.0 | Simple prototyping vector store |
| Qdrant | Vector database | Server (Docker/cloud) | Apache 2.0 | Scaled, multi-client production search |
| LlamaIndex | RAG/orchestration framework | Needs external vector store | MIT | Data connectors atop any vector DB |
Common Mistakes When Evaluating txtai
These mistakes come from applying assumptions about server-based vector databases to a library with a fundamentally different deployment model.
Frequently Asked Questions
Is txtai free to use?
Yes. txtai is open source under the Apache 2.0 license with no usage cap or license fee for the library itself. NeuML, the company that maintains it, sells paid AI consulting services and is separately developing a hosted product called txtai.cloud, still in development at the time of writing.
Does txtai require running a separate database server?
No. txtai embeds its vector index and metadata store directly inside your Python process — by default a Faiss ANN index plus a SQLite file, both persisted as local files with no server process to deploy or monitor.
What ANN backends does txtai support besides Faiss?
Faiss is the default. txtai also supports HNSW, Annoy, and pgvector (plus other backends via its ann extras package), configurable through the backend setting without changing application code.
How is txtai different from Chroma?
Both ship an embedded vector store, but Chroma's typical production path is running as a server, while txtai has no separate server mode to graduate into — it also bundles RAG pipelines, agents, and multi-model workflows in the same package, which Chroma does not.
How is txtai different from Qdrant?
Qdrant is a dedicated vector database service designed to run as its own process (via Docker or a managed cloud endpoint) and be queried by many clients at once. txtai runs embedded inside a single application process, trading that concurrency and horizontal scale for zero deployment overhead.
Does txtai support retrieval-augmented generation (RAG)?
Yes. The RAG pipeline combines an Embeddings index with a local or API-based LLM, retrieves relevant passages for a query, and generates a cited answer — txtai's own documentation frames RAG as more than vector search, also covering web and SQL context retrieval.
Can txtai use local LLMs instead of a cloud API?
Yes. txtai loads models through Hugging Face Transformers, llama.cpp (GGUF format), Ollama, or vLLM for fully local inference, or routes to OpenAI, Anthropic Claude, or AWS Bedrock via LiteLLM when an API-based model is preferred — the same LLM/RAG pipeline interface covers both.
Does txtai support AI agents?
Yes, built on the Hugging Face smolagents framework. txtai agents connect embeddings, pipelines, and workflows together to work through multi-step tasks autonomously, and support agent prompting conventions like agents.md and skill.md.
What license is txtai released under?
Apache License 2.0, which permits commercial use, modification, and redistribution without a royalty, the same permissive license used by Chroma and Qdrant.
Who maintains txtai?
txtai is developed and maintained by NeuML, a company founded by David Mezzetti. NeuML offers paid AI consulting services around the txtai stack alongside maintaining the open-source library.
Can txtai handle large datasets that do not fit on one machine?
Not in its default embedded mode. A single-file Faiss/SQLite index is scoped to the machine that holds it. Datasets that must shard across multiple nodes, or that need many independent services querying one shared index concurrently, are a better fit for a dedicated, horizontally scalable vector database.
Is txtai a good choice for a first RAG prototype?
Yes, for a Python developer specifically — the entire stack (index, RAG pipeline, and optionally an LLM) installs with one pip install txtai and runs in a single script, with no database container to stand up before writing the first line of application logic.