Quick Answers to Local LLM Questions
127 short-answer guides. VRAM requirements, Ollama picks, hardware comparisons and setup tips, answered in 60 seconds or less.
Start here
Four common questions. Pick the one that fits you.
Explore by topic
10 topics, 127 short answers. One click opens a topic right where you are.
Quantization & VRAM 13 Answers
How much memory you need, which quantization format to pick, and VRAM decision trees.
Ollama 7 Answers
Latest versions, best models, context windows, vision, and CPU-only use.
Tool Comparisons 17 Answers
Two-way comparisons: Ollama vs LM Studio, Jan vs LM Studio, Qwen vs DeepSeek.
Model Comparisons 19 Answers
Best 14B models, MoE models, mini PCs, and head-to-head model matchups.
Hardware-Specific 29 Answers
Hardware picks and buying-guide bites: GPU recommendations by budget, mini-PCs, SSDs, cloud GPUs, and eGPUs.
Quick Answers 10 Answers
Yes/no and one-number answers: RAM limits, laptop recommendations.
Prompt Engineering 2 Answers
Quick definitions and best-of lists for prompt engineering concepts.
Privacy & Compliance 7 Answers
GDPR compliance, data sovereignty, and privacy-safe local AI deployment.
US Balcony Solar Regulation 8 Answers
State-by-state legal status, wattage caps, and certification requirements for plug-in solar in the US.
More answers 15 Answers
Short answers that are not sorted into one of the topics above yet.
Why short answers
Every Prompt Bite starts with the answer. Read it in 60 seconds or less, then follow a deep-dive link if you want the long version.
In short: Prompt Bites answer one local LLM question first, in 60 seconds or less, and point to a deeper guide when you need more.
Recently updated
The three pages with the newest change.
Go deeper
The other topic worlds, each in its own color.
Frequently asked questions
How much VRAM do you need for a local LLM?
4 GB VRAM handles Llama 3.2 3B and Phi-3.5-mini comfortably with safe headroom for context expansion. 6 GB runs Llama 3.1 8B at Q4. 12 GB fits Qwen3 14B Q4 efficiently. 16+ GB is needed for 70B models at Q4.
- 4 GB: Llama 3.2 3B Q4, Phi-3.5-mini Q4
- 6 GB: Llama 3.1 8B Q4_K_M
- 8–12 GB: Llama 3.1 8B Q5_K_M, Qwen3 14B Q4
What is Q4_K_M quantization?
Q4_K_M means 4-bit quantization using k-quant (K) compression at medium (M) quality. It is a common default for most models: lower perplexity than Q4_0, smaller than Q8_0.
- Q = quantized, 4 = 4-bit, K = k-quant, M = medium
- Lower perplexity than Q4_0 (6.41 vs 6.70 on Llama 3 8B) for a file 0.24 GiB larger
- Use Q4_K_M as your default quantization
Ollama vs LM Studio: which should you pick?
If you use a terminal and build with APIs, choose Ollama. If you prefer a GUI and just want to chat with models, use LM Studio. Both are free and run models locally.
- Ollama: CLI-first with a desktop app, OpenAI-compatible API, optional cloud models
- LM Studio: desktop GUI with model browser, plus `lms` CLI and headless `llmster`
- Both run on Mac, Windows, and Linux
Best Ollama models right now?
For general use, qwen3.5:9b is the strongest current pick at a 6.6 GB download. For coding, qwen2.5-coder:7b at 4.7 GB does the job on far less hardware than most guides claim. For compact setups, llama3.2:3b runs in 2.0 GB. All sizes read from the Ollama library on 28 August 2026.
- Best general: qwen3.5:9b — 6.6 GB
- Best coding: qwen2.5-coder:7b — 4.7 GB
- Best compact: llama3.2:3b — 2.0 GB
Best mini PC for local LLM?
Three mini PCs stand out for local LLM inference: Mac Mini M4 delivers ~18 tok/s with unified memory and zero VRAM bottleneck, Minisforum UM790 Pro scales to 64 GB DDR5 for larger models, and Beelink SER8 offers value at ~8 tok/s with Ryzen 9 8845HS. All three run 7–13B Q4 models without a discrete GPU.
- Mac Mini M4: fastest for LLMs, ~18 tok/s on Llama 3 8B, power-efficient
- Minisforum UM790 Pro: AMD Radeon 780M iGPU, up to 64 GB unified RAM
- Beelink SER8: Ryzen 9 8845HS, ~8 tok/s, budget-friendly alternative
Which Ollama models support 128K context?
Llama 3.1 8B supports 128K context on Ollama. Qwen3-30B-A3B, a mixture-of-experts model, reaches 256K context. Note: running full context dramatically increases VRAM — a 128K window needs 3–4× more VRAM than the default 4K window.
- Llama 3.1 8B: 128K context, ~16 GB VRAM at full context
- Qwen3-30B-A3B: 256K context, ~24+ GB VRAM at full context
- Set num_ctx 4096 for normal use to save VRAM