Skip to main content
PromptQuorumBuilt for humans. Structured for AI.

Best Ollama Models for 4 GB VRAM?

By · Founder of PromptQuorum · Discovery engine for open-weight & open-source AI

Key Takeaways

  • ✓Phi-4 Mini and Qwen3 4B (2.5 GB each) are the largest models that leave headroom on a 4 GB GPU
  • ✓Most default Ollama tags are already 4-bit (Q4_K_M); check the download size, not the parameter count
  • ✓Llama 3.1 8B (4.9 GB) and Gemma 4 E2B (4.6 GB) do not fit
  • ✓With only 4 GB of system RAM, stay under about 1.5 GB: Qwen3 0.6B, Gemma 3 1B, Llama 3.2 1B
Best Ollama Models for 4 GB VRAM?

Quick Answer

4 GB VRAM fits models up to about 4 billion parameters at 4-bit quantization: Phi-4 Mini and Qwen3 4B (~2.5 GB each), Llama 3.2 3B (~2.0 GB), Qwen3.5 2B (~1.9 GB) and Gemma 3 1B (~0.8 GB). Llama 3.1 8B (~4.9 GB) does not fit.

  • ▸Phi-4 Mini and Qwen3 4B: largest models with headroom (2.5 GB each)
  • ▸Qwen3.5 2B: newest family, text and image input (1.9 GB at q4_K_M)
  • ▸Gemma 3 1B: smallest option (0.8 GB)
Quantization & VRAMIntermediate

What Fits in 4 GB VRAM

On a 4 GB GPU, models up to about 4 billion parameters fit at 4-bit quantization, and the Ollama download size is the number to check. A model needs its file size plus room for the context cache (the KV cache), which grows as the conversation gets longer. As a rule of thumb, keep the file at or below about 3 GB so the whole model stays on the GPU.

Phi-4 Mini and Qwen3 4B are the largest models in the table that leave that headroom, at 2.5 GB each. Qwen3.5 2B is the newest family and accepts images as well as text. Gemma 3 1B and Qwen3 1.7B suit a small file with a long context. File sizes come from the Ollama model library; speed depends on your GPU, so no tokens-per-second figures are quoted here. Newer generations do not replace them at this size: Qwen3.6 starts at 27B (17 GB), and no Gemma 4 download is smaller than 4.3 GB.

ModelOllama tagFile sizeNotes
Phi-4 Miniphi4-mini2.5 GB3.8B parameters, 128K context
Qwen3 4Bqwen3:4b2.5 GB4B parameters, thinking mode
Llama 3.2 3Bllama3.2:3b2.0 GB3B parameters, Llama family
Qwen3.5 2Bqwen3.5:2b-q4_K_M1.9 GBText and image input
Qwen3 1.7Bqwen3:1.7b1.4 GBLeaves most of the VRAM free
Gemma 3 1Bgemma3:1b0.8 GBSmallest option in this table

📍 In One Sentence

On 4 GB of VRAM, Phi-4 Mini (2.5 GB), Qwen3 4B (2.5 GB), Llama 3.2 3B (2.0 GB), Qwen3.5 2B (1.9 GB) and Gemma 3 1B (0.8 GB) fit at 4-bit; Llama 3.1 8B (4.9 GB) does not.

💬 In Plain Terms

If your GPU has 4 GB, pick a model whose download is about 3 GB or less and start with phi4-mini or qwen3:4b. 8B models need a 6 GB card or more.

What Won't Fit in 4 GB

These models are commonly requested, but their default Ollama downloads are close to or above 4 GB:

A 6 GB card makes 8B models such as Llama 3.1 8B practical. See the best local LLMs for 6 GB VRAM. For a full hardware comparison, see fastest local LLMs for low-end PCs.

  • ▸Llama 3.1 8B (llama3.1:8b) — 4.9 GB download, no room left for context
  • ▸Gemma 4 E2B (gemma4:e2b-it-q4_K_M) — 4.6 GB download despite the "2B" name
  • ▸Gemma 4 E4B (gemma4:e4b-it-q4_K_M) — 6.6 GB download
  • ▸Qwen3.5 4B (qwen3.5:4b-q4_K_M) and Gemma 3 4B (gemma3:4b) — 3.3 and 3.4 GB; marginal, a long context pushes layers to system RAM

Which Ollama Models Are 4-Bit?

Most Ollama default tags are already 4-bit: phi4-mini, qwen3:1.7b, gemma3:4b and llama3.1:8b share their file hash with the q4_K_M entry, a 4-bit quantization. The library page often does not name the quantization on the default tag, so for certainty pull the tag with it in the name, for example ollama run qwen3:4b-q4_K_M.

Not every small model defaults to 4-bit, and the default can be much larger than the 4-bit file:

  • ▸gemma2:2b — default is q4_0 (1.6 GB)
  • ▸smollm2:1.7b — default is q8_0 (1.8 GB); the q4_K_M file is 1.1 GB
  • ▸qwen3.5:2b — default is 2.7 GB; qwen3.5:2b-q4_K_M is 1.9 GB
  • ▸llama3.2:1b — default matches q8_0 (1.3 GB)

Related Guides

Quick Answers About 4 GB VRAM Models

Is 4 GB VRAM enough for a useful LLM?▾
Yes, for basic tasks. A 2–4B model such as Phi-4 Mini or Qwen3 4B handles Q&A, short summaries and light coding help. Long documents, multi-step agents and large contexts are limited by the context cache, so 6 GB or more is more comfortable.
Which Ollama model has 4-bit quantization?▾
Most of them. Default tags such as phi4-mini and gemma3:4b share their hash with the q4_K_M (4-bit) entry. Some defaults differ: smollm2 is q8_0 and gemma2:2b is q4_0. Pull the tag with q4_K_M in its name to be sure.
Can I run Llama 3 on 4 GB VRAM?▾
Not the 8B model: llama3.1:8b is a 4.9 GB download. Llama 3.2 3B fits at 2.0 GB if you specifically want a Llama variant. See the full VRAM requirements guide.
What GPU has 4 GB VRAM?▾
The RTX 3050 and 3050 Ti (laptop versions), GTX 1650 Super and AMD RX 6500 XT come with 4 GB. NVIDIA cards need compute capability 5.0 or higher. The RX 6500 XT is not on Ollama's ROCm list, so check the Ollama GPU documentation before buying.
Can CPU-only mode bypass the 4 GB VRAM limit?▾
Partly. When a model is larger than VRAM, Ollama splits its layers between the GPU and system RAM, which still runs but more slowly. With no GPU it runs on the CPU entirely, and speed then depends on your RAM and processor.
What can I run with only 4 GB of system RAM?▾
As a rule of thumb, stay under about 1.5 GB: qwen3:0.6b (0.5 GB), gemma3:1b (0.8 GB), llama3.2:1b (1.3 GB) or qwen3:1.7b (1.4 GB). The operating system and Ollama need part of the 4 GB, so larger models start swapping to disk.
What are the minimum requirements for Ollama?▾
There is no single minimum: the requirement is the model's file size plus the context cache. The smallest models (Gemma 3 270M at 0.3 GB, Qwen3 0.6B at 0.5 GB) run on almost any machine; 3–4B models want about 3 GB of free VRAM or RAM.
Do Gemma 4 and Qwen3.5 fit in 4 GB?▾
Qwen3.5 0.8B and 2B fit comfortably, and the 4B (q4_K_M, 3.3 GB) is marginal. Gemma 4 E2B does not fit: its q4_K_M file is 4.6 GB.
MCP