Skip to main content
PromptQuorumPromptQuorum

Best Embedding Model for Local RAG?

Read in:
🇺🇸en🇩🇪de🇫🇷fr🇯🇵ja🇨🇳zh🇪🇸es🇧🇷pt🇸🇦ar🇰🇷ko

Quick Answer

nomic-embed-text is the best embedding model for local RAG via Ollama — 8,192-token context and efficient on modest hardware. BGE-M3 is the better pick for multilingual document collections.

  • ▸nomic-embed-text: 8,192-token context window, runs via Ollama's embedding API, strong general-purpose choice.
  • ▸BGE-M3: purpose-built for multilingual retrieval — the right pick if your documents span multiple languages.
  • ▸mxbai-embed-large: a strong alternative general-purpose embedding model, slightly larger footprint than nomic-embed-text.

Updated: 2026-07

Quick AnswersIntermediate

Key Takeaways

  • ✓Best general pick: nomic-embed-text — 8,192-token context, runs via Ollama's embedding API
  • ✓Best multilingual pick: BGE-M3 — purpose-built for strong retrieval across many languages
  • ✓Alternative: mxbai-embed-large — strong general-purpose quality, slightly larger footprint
  • ✓Older models like all-MiniLM-L6-v2 are smaller and faster but noticeably lower quality — avoid for serious RAG

Best Pick: nomic-embed-text

nomic-embed-text is the best embedding model for local RAG because it combines a genuinely long 8,192-token context window with efficient inference and native Ollama support through the embedding API. Older embedding models typically cap out around 512 tokens, forcing you to chunk documents more aggressively and losing context across chunk boundaries — nomic-embed-text's longer window reduces that problem.

If your document collection spans multiple languages, BGE-M3 is the better choice. It was trained specifically for strong multilingual retrieval performance, whereas nomic-embed-text and most other options are primarily English-optimized, with weaker performance on non-English text.

mxbai-embed-large is a solid alternative general-purpose pick if you want to compare results against nomic-embed-text — it has a somewhat larger footprint but competitive quality on English-language retrieval benchmarks.

Embedding Models Compared

Avoid older, smaller models like all-MiniLM-L6-v2 for serious RAG work — they are fast and lightweight but noticeably lower quality on retrieval accuracy compared to nomic-embed-text, BGE-M3, or mxbai-embed-large. The quality gap matters more than the speed gain for most RAG use cases.

All three recommended models run through Ollama's embedding API with minimal setup — pulling the model and calling the embeddings endpoint is the same workflow regardless of which one you choose.

Related Reading

Frequently Asked Questions

Do I need a GPU to run an embedding model locally?▾
No — embedding models are much smaller than chat LLMs and run at reasonable speed on CPU. A GPU speeds up bulk indexing of large document collections but isn't required for typical RAG workloads.
Can I use the same model for embeddings and chat?▾
No — embedding models and chat models serve different purposes and use different architectures. Run a dedicated embedding model (like nomic-embed-text) alongside your chat model (like Qwen3 or Llama), not instead of it.
How do I choose a chunk size for RAG with nomic-embed-text?▾
Its 8,192-token context gives flexibility, but shorter chunks (a few hundred tokens) still generally retrieve more precisely than very long chunks, since overly broad chunks dilute the embedding's specificity. Start around 300-500 tokens per chunk and adjust based on retrieval quality.
Is BGE-M3 slower than nomic-embed-text?▾
BGE-M3 is somewhat larger, so expect modestly slower inference and indexing versus nomic-embed-text — a reasonable tradeoff if you need reliable multilingual retrieval.

Related Prompt Bites

← Back to Prompt Bites