Skip to main content
PromptQuorumBuilt for humans. Structured for AI.
Prompt Bites

Quick Answers to Local LLM Questions

127 short-answer guides. VRAM requirements, Ollama picks, hardware comparisons and setup tips, answered in 60 seconds or less.

In one sentence
A Prompt Bite is a short-answer guide to one local LLM question, such as how much VRAM you need or which Ollama model to run.
In plain terms
Every page gives the answer first, in 60 seconds or less. Pick a topic below and open the question you have.
02

Explore by topic

10 topics, 127 short answers. One click opens a topic right where you are.

03

Why short answers

Every Prompt Bite starts with the answer. Read it in 60 seconds or less, then follow a deep-dive link if you want the long version.

Quantization & VRAM
How much memory you need, which quantization format to pick, and VRAM decision trees.
Ollama
Latest versions, best models, context windows, vision, and CPU-only use.
Tool Comparisons
Two-way comparisons: Ollama vs LM Studio, Jan vs LM Studio, Qwen vs DeepSeek.
Hardware-Specific
Hardware picks and buying-guide bites: GPU recommendations by budget, mini-PCs, SSDs, cloud GPUs, and eGPUs.

In short: Prompt Bites answer one local LLM question first, in 60 seconds or less, and point to a deeper guide when you need more.

07

Frequently asked questions

How much VRAM do you need for a local LLM?

4 GB VRAM handles Llama 3.2 3B and Phi-3.5-mini comfortably with safe headroom for context expansion. 6 GB runs Llama 3.1 8B at Q4. 12 GB fits Qwen3 14B Q4 efficiently. 16+ GB is needed for 70B models at Q4.

  • 4 GB: Llama 3.2 3B Q4, Phi-3.5-mini Q4
  • 6 GB: Llama 3.1 8B Q4_K_M
  • 8–12 GB: Llama 3.1 8B Q5_K_M, Qwen3 14B Q4
What is Q4_K_M quantization?

Q4_K_M means 4-bit quantization using k-quant (K) compression at medium (M) quality. It is a common default for most models: lower perplexity than Q4_0, smaller than Q8_0.

  • Q = quantized, 4 = 4-bit, K = k-quant, M = medium
  • Lower perplexity than Q4_0 (6.41 vs 6.70 on Llama 3 8B) for a file 0.24 GiB larger
  • Use Q4_K_M as your default quantization
Ollama vs LM Studio: which should you pick?

If you use a terminal and build with APIs, choose Ollama. If you prefer a GUI and just want to chat with models, use LM Studio. Both are free and run models locally.

  • Ollama: CLI-first with a desktop app, OpenAI-compatible API, optional cloud models
  • LM Studio: desktop GUI with model browser, plus `lms` CLI and headless `llmster`
  • Both run on Mac, Windows, and Linux
Best Ollama models right now?

For general use, qwen3.5:9b is the strongest current pick at a 6.6 GB download. For coding, qwen2.5-coder:7b at 4.7 GB does the job on far less hardware than most guides claim. For compact setups, llama3.2:3b runs in 2.0 GB. All sizes read from the Ollama library on 28 August 2026.

  • Best general: qwen3.5:9b — 6.6 GB
  • Best coding: qwen2.5-coder:7b — 4.7 GB
  • Best compact: llama3.2:3b — 2.0 GB
Best mini PC for local LLM?

Three mini PCs stand out for local LLM inference: Mac Mini M4 delivers ~18 tok/s with unified memory and zero VRAM bottleneck, Minisforum UM790 Pro scales to 64 GB DDR5 for larger models, and Beelink SER8 offers value at ~8 tok/s with Ryzen 9 8845HS. All three run 7–13B Q4 models without a discrete GPU.

  • Mac Mini M4: fastest for LLMs, ~18 tok/s on Llama 3 8B, power-efficient
  • Minisforum UM790 Pro: AMD Radeon 780M iGPU, up to 64 GB unified RAM
  • Beelink SER8: Ryzen 9 8845HS, ~8 tok/s, budget-friendly alternative
Which Ollama models support 128K context?

Llama 3.1 8B supports 128K context on Ollama. Qwen3-30B-A3B, a mixture-of-experts model, reaches 256K context. Note: running full context dramatically increases VRAM — a 128K window needs 3–4× more VRAM than the default 4K window.

  • Llama 3.1 8B: 128K context, ~16 GB VRAM at full context
  • Qwen3-30B-A3B: 256K context, ~24+ GB VRAM at full context
  • Set num_ctx 4096 for normal use to save VRAM
MCP