Best Ollama Models for 4 GB VRAM?

What Fits in 4 GB VRAM

As of May 2026, at 4 GB VRAM you are limited to models with 3 billion parameters or fewer at Q4 quantization. This rules out every mainstream local model — Llama 3 8B, Mistral 7B, Qwen 14B. Three modern small models perform surprisingly well: Phi-4 Mini matches GPT-3.5 on instruction following, Gemma 2 2B handles fast chat, and SmolLM 1.7B runs on integrated graphics.

Phi-4 Mini is the top pick at this tier. Despite its small size, it handles general Q&A, light coding, and document summarization at ~25 tokens per second. Gemma 2 2B is faster for single-turn chat. SmolLM 1.7B is the fallback if even Phi-4 Mini pushes your VRAM too close to the limit.

What Won't Fit in 4 GB

These models are commonly requested but require more than 4 GB VRAM at every quantization level:

Upgrading to 6 GB unlocks Llama 3 8B and Mistral 7B — the two most popular local models. See the best local LLMs for 6 GB VRAM. For a full hardware comparison, see fastest local LLMs for low-end PCs.

• ▸Llama 3 8B — needs ~5.5 GB at Q4_K_M (minimum)
• ▸Mistral 7B — needs ~4.5 GB at Q4_K_M (marginal; risky at 4 GB with context overhead)
• ▸Phi-4 (full 14B) — needs ~9.8 GB
• ▸Qwen 14B — needs ~9.5 GB at Q4_K_M