Skip to main content
PromptQuorumBuilt for humans. Structured for AI.

Best Ollama Models for CPU Only?

By · Founder of PromptQuorum · Discovery engine for open-weight & open-source AI

Key Takeaways

  • ✓CPU inference is 5–10× slower than GPU — expect 3–6 tok/s for a 7B model on a modern 8-core desktop CPU
  • ✓Fastest on CPU: Qwen3 0.6B (~35 tok/s) and Gemma 3 1B (~19 tok/s) in an independent test on an older i5 laptop
  • ✓Best quality on CPU: Phi-4 Mini Q4, about 4 GB RAM, ~7 tok/s in the same test
  • ✓CPU inference is practical for batch jobs and single-query lookups; too slow for interactive chat
Best Ollama Models for CPU Only?

Quick Answer

For the fastest CPU-only Ollama model, use Qwen3 0.6B (~35 tok/s in an independent test) or Gemma 3 1B (~19 tok/s). For the best answer quality without a GPU, use Phi-4 Mini Q4 (~7 tok/s, about 4 GB RAM) or Llama 3.2 3B Q4. Machines with 2 GB RAM should stay at 1B parameters or smaller.

  • ▸Fastest: Qwen3 0.6B (~35 tok/s) and Gemma 3 1B (~19 tok/s), both fit in 2 GB RAM
  • ▸Best quality: Phi-4 Mini Q4 (~7 tok/s), needs about 4 GB RAM
  • ▸Balanced: Llama 3.2 3B Q4, 2.0 GB download, about 4 GB RAM
OllamaIntermediate

The CPU Speed Reality

CPU inference runs at 3–6 tokens per second on a modern 8-core desktop CPU — roughly 5–10× slower than a mid-range GPU. A 7B model at Q4 produces one word approximately every 200–300 milliseconds on CPU.

This speed is acceptable for two use cases: overnight batch processing such as summarizing documents or classifying data, and single-query lookups where a 30-second wait is acceptable. For interactive chat or real-time code completion, CPU inference is too slow to be practical.

The root constraint is memory bandwidth, not CPU clock speed. Consumer CPUs read RAM at 40–80 GB/s. A dedicated GPU reads VRAM at 400–900 GB/s. LLM inference scales directly with memory bandwidth — which is why even a mid-range GPU produces dramatically faster inference than a high-end CPU.

Top 4 Models for CPU-Only Use

Qwen3 0.6B and Gemma 3 1B are the fastest CPU-only Ollama models; Phi-4 Mini Q4 gives the best answer quality at about 7 tok/s. The speeds below come from an independent CPU-only test by It's FOSS on an older Intel i5 laptop with 12 GB RAM; CPUs with faster memory run faster.

Gemma 3 1B (806 MB download) and Qwen3 0.6B (523 MB) are the practical options when RAM is limited to 2 GB, at the cost of weaker multi-step reasoning. Larger models such as Gemma 4 E2B (4.6 GB+ download, ~10 tok/s in the same test) need more memory than a 4 GB machine offers.

For the full breakdown of CPU-only configurations including RAM requirements and OS-level optimizations, see the best CPU-only LLM guide.

ModelOllama downloadMin. RAMCPU speed
Qwen3 0.6B523 MB~2 GB~34–36 tok/s
Gemma 3 1B806 MB~2 GB~19 tok/s
Llama 3.2 3B Q42.0 GB~4 GBNot tested
Phi-4 Mini Q42.5 GB~4 GB~7 tok/s

Related Guides

Quick Answers About CPU-Only LLMs

Which Ollama model is fastest on CPU?▾
Qwen3 0.6B is the fastest, at about 34–36 tok/s in an independent test on an older Intel i5 laptop, followed by Gemma 3 1B at about 19 tok/s. Both trade reasoning depth for speed; use Phi-4 Mini Q4 (~7 tok/s) when answer quality matters more.
Can Ollama run on a PC with 2 GB of RAM?▾
Yes, with a model of 1B parameters or smaller: Qwen3 0.6B (523 MB download) or Gemma 3 1B (806 MB). Close other applications first, because the OS and Ollama runtime also need 1–2 GB. Models of 3B parameters and above need about 4 GB.
How much RAM do I need for CPU-only Ollama?▾
About 2 GB for Qwen3 0.6B or Gemma 3 1B, and about 4 GB for Llama 3.2 3B Q4 or Phi-4 Mini Q4. Add 1–2 GB on top of the download size for the OS and Ollama runtime overhead.
Why is CPU inference so much slower than GPU?▾
LLM inference is memory-bandwidth-bound. Consumer CPUs read RAM at 40–80 GB/s. A mid-range GPU reads VRAM at 400–900 GB/s. That 10× bandwidth difference translates directly into 5–10× slower token generation.
Can I use Ollama on a laptop without a dedicated GPU?▾
Yes. Ollama runs on CPU automatically when no GPU is detected. Expect 3–5 tok/s on a modern laptop CPU. See the best Ollama models right now for GPU-tier recommendations if you later upgrade.
Which CPUs are fastest for local LLM inference?▾
Apple M-series chips (M3, M4) use unified memory architecture and reach 15–30 tok/s on 7B models — far exceeding x86 CPUs on CPU-only inference. Among x86 CPUs, those with higher memory bandwidth and large L3 cache perform best.
MCP