Best Ollama Models for CPU Only?
By Hans Kuepper · Founder of PromptQuorum · Discovery engine for open-weight & open-source AI
Key Takeaways
- ✓CPU inference is 5–10× slower than GPU — expect 3–6 tok/s for a 7B model on a modern 8-core desktop CPU
- ✓Fastest on CPU: Qwen3 0.6B (~35 tok/s) and Gemma 3 1B (~19 tok/s) in an independent test on an older i5 laptop
- ✓Best quality on CPU: Phi-4 Mini Q4, about 4 GB RAM, ~7 tok/s in the same test
- ✓CPU inference is practical for batch jobs and single-query lookups; too slow for interactive chat

Quick Answer
For the fastest CPU-only Ollama model, use Qwen3 0.6B (~35 tok/s in an independent test) or Gemma 3 1B (~19 tok/s). For the best answer quality without a GPU, use Phi-4 Mini Q4 (~7 tok/s, about 4 GB RAM) or Llama 3.2 3B Q4. Machines with 2 GB RAM should stay at 1B parameters or smaller.
- ▸Fastest: Qwen3 0.6B (~35 tok/s) and Gemma 3 1B (~19 tok/s), both fit in 2 GB RAM
- ▸Best quality: Phi-4 Mini Q4 (~7 tok/s), needs about 4 GB RAM
- ▸Balanced: Llama 3.2 3B Q4, 2.0 GB download, about 4 GB RAM
The CPU Speed Reality
CPU inference runs at 3–6 tokens per second on a modern 8-core desktop CPU — roughly 5–10× slower than a mid-range GPU. A 7B model at Q4 produces one word approximately every 200–300 milliseconds on CPU.
This speed is acceptable for two use cases: overnight batch processing such as summarizing documents or classifying data, and single-query lookups where a 30-second wait is acceptable. For interactive chat or real-time code completion, CPU inference is too slow to be practical.
The root constraint is memory bandwidth, not CPU clock speed. Consumer CPUs read RAM at 40–80 GB/s. A dedicated GPU reads VRAM at 400–900 GB/s. LLM inference scales directly with memory bandwidth — which is why even a mid-range GPU produces dramatically faster inference than a high-end CPU.
Top 4 Models for CPU-Only Use
Qwen3 0.6B and Gemma 3 1B are the fastest CPU-only Ollama models; Phi-4 Mini Q4 gives the best answer quality at about 7 tok/s. The speeds below come from an independent CPU-only test by It's FOSS on an older Intel i5 laptop with 12 GB RAM; CPUs with faster memory run faster.
Gemma 3 1B (806 MB download) and Qwen3 0.6B (523 MB) are the practical options when RAM is limited to 2 GB, at the cost of weaker multi-step reasoning. Larger models such as Gemma 4 E2B (4.6 GB+ download, ~10 tok/s in the same test) need more memory than a 4 GB machine offers.
For the full breakdown of CPU-only configurations including RAM requirements and OS-level optimizations, see the best CPU-only LLM guide.
| Model | Ollama download | Min. RAM | CPU speed |
|---|---|---|---|
| Qwen3 0.6B | 523 MB | ~2 GB | ~34–36 tok/s |
| Gemma 3 1B | 806 MB | ~2 GB | ~19 tok/s |
| Llama 3.2 3B Q4 | 2.0 GB | ~4 GB | Not tested |
| Phi-4 Mini Q4 | 2.5 GB | ~4 GB | ~7 tok/s |
Related Guides
- ▸Radeon 6800M for Local LLM: Full Setup Guide -- Radeon GPU guide
- ▸Strix Halo + Ollama + Vulkan: Performance Guide -- Strix Halo guide
Quick Answers About CPU-Only LLMs
Which Ollama model is fastest on CPU?▾
Can Ollama run on a PC with 2 GB of RAM?▾
How much RAM do I need for CPU-only Ollama?▾
Why is CPU inference so much slower than GPU?▾
Can I use Ollama on a laptop without a dedicated GPU?▾
Which CPUs are fastest for local LLM inference?▾
Want the full breakdown?
Read the complete guide →Related Prompt Bites