Skip to main content
PromptQuorumBuilt for humans. Structured for AI.

Best Local LLM for Coding with 12 GB VRAM?

By · Founder of PromptQuorum · Discovery engine for open-weight & open-source AI

Key Takeaways

  • ✓Gemma 4 12B (Google, Apache 2.0): 7.7–8.0 GB download, 256K context, 72.0% LiveCodeBench v6 (vendor-reported) — the top coding pick for any 12 GB GPU
  • ✓Qwen2.5-Coder 14B Q4_K_M (~9 GB) is still a solid code-specialised option, but it dates from 2024; newer Qwen coders such as Qwen3.8 27B (18 GB) need more than 12 GB
  • ✓Devstral Small 2 (24B, 15 GB) and Qwen3.8 27B (18 GB) only run on 12 GB with CPU offload, which slows generation sharply
  • ✓Raise the context window to 8192 or more for coding — the default window truncates real source files — and confirm with ollama ps that the model sits fully on the GPU
Best Local LLM for Coding with 12 GB VRAM?

Quick Answer

Gemma 4 12B is the best coding model for a 12 GB VRAM GPU: its Ollama download is 7.7–8.0 GB, and Google reports 72.0% on LiveCodeBench v6 for the 12B model. Qwen2.5-Coder 14B (~9 GB) is the code-specialised alternative. Qwen3.8 27B (18 GB) and Devstral Small 2 (15 GB) do not fit in 12 GB.

  • ▸Gemma 4 12B: 7.7–8.0 GB download, 256K context, Apache 2.0
  • ▸Qwen2.5-Coder 14B Q4_K_M: ~9 GB, code-specialised, supports fill-in-the-middle
  • ▸Ornith 9B: 5.6 GB, MIT licence, built for agentic coding
Hardware-SpecificIntermediate

The Coding Pick for 12 GB

Gemma 4 12B is the best coding model for a 12 GB VRAM GPU: the Ollama download is 7.7–8.0 GB, which leaves about 4 GB for the KV cache, and Google reports 72.0% on LiveCodeBench v6 and a Codeforces Elo of 1659 for the 12B model. It is licensed under Apache 2.0, supports a 256K context window and accepts image input, so you can paste a screenshot of an error or a UI bug. These benchmark figures are vendor-reported; we did not run our own.

Qwen2.5-Coder 14B Q4_K_M (~9 GB) still fits and remains the code-specialised choice with fill-in-the-middle support, but it is a 2024 model. Ornith 9B (5.6 GB, MIT licence) targets agentic coding and leaves the most headroom for long context. Devstral Small 2 (24B, 15 GB) and Qwen3.8 27B (18 GB) exceed 12 GB and only run with CPU offload.

If you are on an 8 GB card instead, see the best LLMs for AMD 5700X + RTX 3070 Ti for the 8 GB compromise options.

ModelDownload sizeBest for
Gemma 4 12B7.7–8.0 GBTop pick: coding, images, 256K context
Qwen2.5-Coder 14B Q4_K_M~9 GBCode-specialised, fill-in-the-middle
Ornith 9B5.6 GBAgentic coding with long-context headroom
Devstral Small 2 24B15 GBDoes not fit; needs CPU offload
Qwen3.8 27B18 GBDoes not fit; needs 18 GB+ memory

Settings That Matter for Coding

Set the context window to at least 8k for coding work — a short default window truncates source files of a few hundred lines. Gemma 4 12B's ~8 GB of weights leave roughly 4 GB of a 12 GB card for the KV cache and runtime; Qwen2.5-Coder 14B at ~9 GB leaves less, so keep its context moderate. Set it with OLLAMA_CONTEXT_LENGTH=8192 before starting the server, or /set parameter num_ctx 8192 inside ollama run, then run ollama ps to confirm the model is on the GPU and not split with the CPU.

Enable Flash Attention (OLLAMA_FLASH_ATTENTION=1) and, if you need more context, set OLLAMA_KV_CACHE_TYPE=q8_0, which roughly halves KV-cache memory compared with the default f16 cache at a small quality cost. The environment variables can be combined in one launch.

For the full ranking beyond 12 GB, see the best local LLMs for coding guide.

ollama pull gemma4:12b
ollama run gemma4:12b

# code-specialised alternative
ollama pull qwen2.5-coder:14b-instruct-q4_K_M
Memory bandwidth, not VRAM size, sets generation speed: the RTX 3080 Ti 12 GB (912 GB/s) generates tokens noticeably faster than the RTX 3060 12 GB (360 GB/s), although both hold the same models. We did not benchmark tokens per second for this page.

Related Guides

Quick Answers About Coding LLMs for 12 GB VRAM

What is the best coding model for 12 GB VRAM?▾
Gemma 4 12B. Its Ollama download is 7.7–8.0 GB, it supports a 256K context window, and Google reports 72.0% on LiveCodeBench v6 and a Codeforces Elo of 1659 for it. Qwen2.5-Coder 14B Q4_K_M (~9 GB) is the alternative if you want a code-only model with fill-in-the-middle support.
What is the best local LLM for 12 GB VRAM in general?▾
Gemma 4 12B again: it is a general-purpose multimodal model (text and image input) under Apache 2.0, so one 8 GB download covers chat, coding and screenshots. Ollama also lists a lighter E4B variant if you want a smaller footprint.
Do Qwen3.8 27B and Devstral Small 2 fit in 12 GB of VRAM?▾
No. Ollama lists qwen3.8:27b at 18 GB and devstral-small-2:24b at 15 GB. Both can run with part of the model offloaded to system RAM, but generation slows sharply once layers leave the GPU, so they suit 16–24 GB cards.
Should I use Q4_K_M or a higher quantization on 12 GB?▾
Use Q4_K_M for 14B-class models — it is the largest standard quantization that leaves room for context on 12 GB. Smaller models such as Ornith 9B (5.6 GB default) leave headroom, so a higher-precision tag is possible where the model page lists one.
Can I use fill-in-the-middle (FIM) autocomplete with these models on Ollama?▾
Qwen2.5-Coder 14B supports FIM. Ollama accepts the text after the cursor in the suffix field of /api/generate for models whose template supports it. Check each model's Ollama page for insert support before wiring it into an editor; for autocomplete, Qwen2.5-Coder is the safer choice.
MCP