Skip to main content
PromptQuorumBuilt for humans. Structured for AI.
Local LLMs

Best Local LLMs by VRAM Tier 2026: Ollama, LM Studio & Hardware Guide

Local LLMs are large language models that run entirely on your own computer — no internet connection, no API fees, and no data leaving your machine. Compared with the OpenAI API you trade a little quality for privacy and zero running cost. If you are just starting, install Ollama and run Llama 3.2 3B or Qwen3.5 4B on 8 GB of RAM in under 10 minutes; the tier list below is a ranking of the best models, GPUs, and tools by VRAM — 4GB, 12GB, 24GB, 32GB and 48GB — for every budget.

In one sentence
Local LLMs are language models that run entirely on your own computer: no internet connection, no API fees, and no data leaving your machine.
In plain terms
Install Ollama or LM Studio, download a model that fits your VRAM, and start chatting. The guides below cover every step, from the first install to hardware, privacy and cost.
02

Explore by topic

15 topics, 174 guides. One click opens a topic right where you are.

03

Practical answers

Four practical answers, calculated with a simple memory formula (estimates, not measurements). The full tables are at the bottom of this page.

Largest LLM You Can Run Locally
Memory sets the limit, not the software: at Q4 a model needs about 0.6 GB per billion parameters plus 1–2 GB of headroom. One 24 GB card runs a 27B-class model; 48 GB runs a 70B model.
Best Quantization for 16 GB of VRAM or RAM
Quantized size is roughly parameters × 0.6 GB at Q4, × 0.7 at Q5 and × 1.05 at Q8, plus 1–2 GB for context. On 16 GB of VRAM, Q5 is a good default for 12–14B models and a 22B model fits only at Q4. With 16 GB of system RAM and no GPU, halve the budget: a 12B model at Q4 (about 7.2 GB) is the comfortable choice.
Best Models for a Mid-Range PC in LM Studio
A mid-range PC means 12–16 GB of VRAM, for example an RTX 3060 12 GB or RTX 4060 Ti 16 GB. In LM Studio, search for the model, take the Q4 file if its size plus 1–2 GB fits your VRAM, and step up to Q5 when there is room.
Local RAG on a Low-RAM Laptop
RAG needs two small models: an embedding model (nomic-embed-text, 137M parameters) and a chat model. On a CPU-only machine the chat model needs about twice its Q4 GPU size, so a 1B model takes about 1.2 GB and a 3B model about 3.6 GB. A 2 GB laptop is the floor, with almost no headroom for the operating system and your index; 8 GB is the practical starting point.

In short: 8 GB of RAM is enough for a 7B model, all data stays on your machine, and there are no API costs once you own the hardware.

06

Overview as a presentation

Best Local LLMs by VRAM Tier 2026: Ollama, LM Studio & Hardware Guide

The slide deck below covers hardware requirements (8 GB VRAM for 7B models, about 42 GB for a 70B model), top open-source models 2026, Ollama setup in 5 minutes, Q4_K_M quantization, regional compliance (GDPR, APPI), and key takeaways. Download the PDF as a Local LLMs quick-reference card.

07

Frequently Asked Questions

What is a local LLM?

A large language model (e.g., Qwen3.8, Gemma 4, gpt-oss, DeepSeek) that runs on your own hardware instead of a cloud API. Prompts and files stay on your machine, it works offline, has no usage limits, and costs nothing per token after the hardware purchase.

How much VRAM do I need for a local LLM?

8 GB of VRAM runs 7–8B models at Q4 quantization (about 5 GB of weights). 16 GB handles 13–14B models comfortably. A 70B model needs about 42 GB for its Q4 weights, so a 48 GB setup (for example two RTX 4090s) is the practical minimum. Apple Silicon unified memory counts as VRAM, minus the 8 GB macOS keeps for itself.

What is the difference between Ollama and LM Studio?

Ollama is a CLI tool that runs models via simple terminal commands and exposes an OpenAI-compatible API at `localhost:11434`. LM Studio provides a desktop GUI, model browser, and built-in chat interface. Both support the same models.

Can local LLMs match cloud models like GPT or Claude?

Open-weight models such as Qwen3.8-27B and Gemma 4 handle everyday coding, writing and summarizing on a single 24 GB GPU. Leading cloud models still have an edge on complex multi-step reasoning. Vendor-reported benchmark scores differ by test, so try your own prompts before choosing.

How do I fine-tune a local model?

Fine-tuning requires 500+ labeled training examples, the QLoRA framework (reduces VRAM requirement via 4-bit quantization), 24 GB+ VRAM (or a cloud GPU rental), and 1–4 hours of training time for a 7B model.

What is the minimum hardware to run a local LLM in 2026?

Minimum: 8 GB RAM and any modern CPU (runs 3B–7B models at roughly 2–5 tokens/sec). Recommended: a GPU with 8 GB+ VRAM (RTX 3060 or newer) for roughly 15–40 tokens/sec on 7–8B models.

Are local LLMs free to use?

Mostly yes. Ollama is open source and LM Studio is free to use but closed source. Most open-weight models (Qwen, Gemma, gpt-oss, Mistral, DeepSeek) are free to download, but licenses differ and Llama uses a community license, so check the terms before commercial use. The main cost is your hardware.

What is the best local LLM for coding in 2026?

Qwen3.8 27B is the top pick for a 24 GB GPU (18 GB, 256K context). Devstral Small 2 (24B, 15 GB) suits 16 GB machines, and Qwen3 8B suits 8 GB. Codestral 22B remains a common choice for IDE autocomplete, and Phi-4 Mini is the practical CPU-only choice. Benchmark scores are vendor-reported; see the coding guide for details.

Can I run a local LLM without a GPU?

Yes. Any modern CPU can run 3B–7B models at Q4 quantization using Ollama (CPU mode) or LM Studio. Typical speed is roughly 2–8 tokens/sec for 7B models on a laptop CPU and faster for 3–4B models, compared with 20–50 tokens/sec on an RTX 4060. 7B Q4 needs about 5 GB of RAM (not VRAM). For CPU-only setups, Phi-4 Mini (3.8B) and Llama 3.2 3B offer the best quality-to-speed ratio.

How do I update local LLM models when new versions are released?

Ollama: run `ollama pull <model-name>` again — it downloads only changed layers. LM Studio: open the model browser, find the updated version, and download it. Old GGUF files are not automatically removed — delete them manually from ~/.ollama/models (Ollama) or ~/Library/Application Support/LM Studio/models (macOS) to free disk space. New models usually appear in the Ollama library within days of an official release.

What are the best Ollama models in 2026?

Top picks in the Ollama library: Qwen3.8-27B (our pick for a 24 GB card, `ollama pull qwen3.8:27b`, 18 GB), gpt-oss:20b (small, 16 GB class), Gemma 4 (vision and tool calling, E2B to 31B; `ollama pull gemma4:12b` is about 8 GB), Qwen3 8B (`ollama pull qwen3:8b`, about 5 GB) for 8 GB machines, and DeepSeek-R1 distills for step-by-step reasoning. Run any model with `ollama run <name>` after pulling.

What is the best local LLM for an RTX 3060 12 GB VRAM?

The RTX 3060 12 GB is a good local LLM GPU. Best fits: Qwen3 14B (about 9 GB, 9–12 tokens/sec), Gemma 4 12B (about 8 GB, 11–14 tokens/sec), Qwen3 8B (16–20 tokens/sec) or DeepSeek-R1 7B (10–12 tokens/sec). Llama 4 Scout does not fit: it needs about 67 GB. The 12 GB of VRAM also puts you above the RTX 3060 Ti (8 GB) and opens up 13–14B-class models at full quality.

Ollama vs LM Studio vs Jan.ai: which should I use?

Use Ollama if you want a CLI tool with an OpenAI-compatible API at localhost:11434 — best for developers and automation. Use LM Studio if you want a desktop GUI, built-in model browser, and chat interface — best for beginners. Use Jan.ai if you want a privacy-focused chat app with a built-in model store. All three support the same GGUF models. Setup time: Ollama 2 min, LM Studio 5 min, Jan.ai 5 min.

What are the best budget GPUs for local LLMs in 2026?

According to our budget GPU guide: a used RTX 3060 12 GB (about $270–300) is the cheapest card worth buying, running Qwen3 14B at 9–12 tokens/sec and Qwen3 8B at 16–20 tokens/sec. New alternatives are the RTX 5060 Ti 16 GB (about $570–600) and the Intel Arc B580 12 GB (about $250–290). Prices rose in 2026 because of a memory shortage, so check the guide for current figures. Minimum recommended: 8 GB VRAM for useful 7B inference.

Is a local LLM better than ChatGPT?

For privacy and cost, yes. For raw output quality on the hardest tasks, usually no: leading cloud models still outperform locally runnable models on complex reasoning. For everyday tasks, a 27B-class local model on a 24 GB GPU is a capable alternative at no per-query cost.

How much RAM do I need to run a local LLM?

Minimum: 8 GB RAM to run a 7B model at Q4 quantization. Recommended: 16 GB for 13B models and about 48 GB for 70B models (about 42 GB of Q4 weights plus headroom). On a Mac, unified memory is shared with macOS, which needs at least 8 GB, so subtract 8 GB from the total — a Mac with 24 GB has about 16 GB usable and runs a 13B model well, and a 70B model at Q4 needs a 64 GB Mac. GPU VRAM is equivalent to RAM for GPU inference.

How do I run a local LLM?

Install Ollama (ollama.com), then run one command: `ollama run llama3.1:8b`. The model downloads automatically and you can start chatting in under 5 minutes. No API key, no account, no internet connection after the initial download.

What is the best free local LLM in 2026?

For a 24 GB GPU: Qwen3.8-27B (18 GB, `ollama pull qwen3.8:27b`). For 16 GB: gpt-oss:20b or Gemma 4 12B. For 8 GB: Qwen3 8B. For CPU-only setups: Phi-4 Mini. All are open-weight and free to download via `ollama pull`; check each license before commercial use.

Are local LLMs private?

Yes, for inference. When running with Ollama or LM Studio, prompts, documents and responses are processed on your machine and are not sent to a model provider's server. Update checks and optional telemetry are separate, so review each app's settings. Local processing suits confidential and personal data and simplifies GDPR work, but legal compliance still depends on your whole setup.

Key Takeaways

  • 8 GB RAM is enough to run a 7B model locally (Ollama or LM Studio, under 10 min setup)
  • 48 GB of VRAM runs a 70B model at Q4 (about 42 GB of weights); Llama 4 Scout (109B MoE) needs about 67 GB
  • Q4 quantization cuts memory to roughly a quarter of FP16 with a small quality loss — a 7B model fits in 4–5 GB VRAM
  • Open-weight models such as Qwen3.8, Gemma 4 and gpt-oss now cover most everyday coding and writing tasks; leading cloud models still lead on the hardest multi-step work
  • Zero API costs after hardware purchase — no usage limits, no vendor lock-in
  • Prompts and files stay on your machine, with no cloud storage — check each app's telemetry setting; local processing simplifies GDPR work but does not replace it
  • LoRA fine-tuning requires 500+ labeled examples and 24 GB+ VRAM (or cloud GPU for training)
  • Qwen Local Deployment Guide 2026 — one-command Ollama setup for Qwen 3.6 27B and Qwen3 8B, plus Coder and VL tiers
  • Best GPU Under $500 for LLM Inference — Intel Arc B580 12 GB leads; a used RTX 3060 12 GB is the CUDA runner-up
  • DeepSeek vs Qwen: Local Comparison 2026 — benchmark head-to-head
  • Alibaba Cloud vs Tencent Cloud GPU 2026 — GPU cloud for the Chinese market
  • Local LLM Cost Calculator: Build vs Rent 2026 — 3-year ROI calculator

Newest Models Added

ModelPull CommandDownload sizeNotes
Qwen3.8-27Bollama pull qwen3.8:27b18 GBAlibaba. 256K context, text and image input; our pick for 24 GB cards
Gemma 4 12Bollama pull gemma4:12b8 GBGoogle. 256K context, vision and tool calling; fits a 12 GB GPU
gpt-oss 20Bollama pull gpt-oss:20b14 GBOpenAI. 128K context, open weights; fits a 16 GB GPU
Qwen3.5 4Bollama pull qwen3.5:4b4 GBAlibaba. Text and image input; our small-model pick for 8 GB machines

Best Local LLM by VRAM Tier

Already know your GPU's VRAM? Find your tier below for the best-fit model — for the full formula, quantization math, and MoE exceptions, see the complete VRAM guide.

VRAMBest-Fit Model (Q4)Example GPUs
4 GBLlama 3.2 3B or Phi-4 Mini (~2–2.5 GB)RTX 3050 4 GB, GTX 1650
8 GBLlama 3.1 8B or Qwen3 8B (~4-5 GB)RTX 4060 Ti 8 GB, RTX 3070
12 GBQwen3 14B (~8.4 GB), or 7-8B at Q5/Q8 for higher qualityRTX 3060 12 GB, RTX 4070
16 GBQwen3 14B at Q5/Q8, or 22B-class (~13 GB)RTX 4060 Ti 16 GB, RTX 4080
24 GBQwen3.8-27B (~18 GB, our top pick for 24 GB cards)RTX 4090, RTX 3090
32 GB22-32B at full precision, or one card in a dual-GPU 70B rigRTX 5090
48 GB70B models at Q4 (~42 GB); Q5 (~50 GB) does not fit2x RTX 4090, RTX 6000 Ada

Largest LLM You Can Run Locally

Memory sets the limit, not the software: at Q4 a model needs about 0.6 GB per billion parameters plus 1–2 GB of headroom. One 24 GB card runs a 27B-class model; 48 GB runs a 70B model.

ModelQuantizationWeightsFits
Qwen3.8-27BQ4~18 GB24 GB VRAM
70B (Llama 3.3, Qwen)Q4~42 GB48 GB VRAM
70BQ5~50 GBAbove a single 48 GB card
70BQ8~74 GBMulti-GPU setup

Short on VRAM? See how to run 70B models on 24 GB VRAM and the full VRAM guide.

Best Quantization for 16 GB of VRAM or RAM

Quantized size is roughly parameters × 0.6 GB at Q4, × 0.7 at Q5 and × 1.05 at Q8, plus 1–2 GB for context. On 16 GB of VRAM, Q5 is a good default for 12–14B models and a 22B model fits only at Q4. With 16 GB of system RAM and no GPU, halve the budget: a 12B model at Q4 (about 7.2 GB) is the comfortable choice.

Model sizeQ4Q5Q8Fits 16 GB VRAM
8B~4.8 GB~5.6 GB~8.4 GBQ4, Q5, Q8
12B~7.2 GB~8.4 GB~12.6 GBQ4, Q5, Q8
14B~8.4 GB~9.8 GB~14.7 GBQ4, Q5
22B~13.2 GB~15.4 GB~23.1 GBQ4

For the quality trade-off between bit levels, see LLM quantization explained.

Best Models for a Mid-Range PC in LM Studio

A mid-range PC means 12–16 GB of VRAM, for example an RTX 3060 12 GB or RTX 4060 Ti 16 GB. In LM Studio, search for the model, take the Q4 file if its size plus 1–2 GB fits your VRAM, and step up to Q5 when there is room.

VRAMModel to tryApprox. size
12 GBQwen3 14B at Q4 (or a 12B-class model)~8.4 GB (12B: ~7.2 GB)
16 GBQwen3 14B at Q5~9.8 GB
16 GB22B-class model at Q4~13.2 GB

Setup: how to install LM Studio. Not sure which app? Read Ollama vs LM Studio.

Local RAG on a Low-RAM Laptop

RAG needs two small models: an embedding model (nomic-embed-text, 137M parameters) and a chat model. On a CPU-only machine the chat model needs about twice its Q4 GPU size, so a 1B model takes about 1.2 GB and a 3B model about 3.6 GB. A 2 GB laptop is the floor, with almost no headroom for the operating system and your index; 8 GB is the practical starting point.

System RAMChat model (Q4)Memory (CPU-only)
2 GBLlama 3.2 1B + nomic-embed-text~1.2 GB for the chat model, very tight
8 GBLlama 3.2 3B + nomic-embed-text~3.6 GB
16 GB7–8B model~8.4–9.6 GB

Figures are estimates from the VRAM formula, not benchmarks. Full setup: local RAG guide and best local RAG tools.

Ollama vs LM Studio vs Jan.ai: Which Should You Use?

FeatureOllamaLM StudioJan.ai
InterfaceTerminal (CLI)Desktop GUIDesktop GUI + chat
API endpointlocalhost:11434localhost:1234localhost:1337
Model browserCLI onlyBuilt-inBuilt-in
Best forDevelopers, automationBeginners, GUI usersPrivacy-first chat
Setup time2 min5 min5 min
VRAM requirements at Q4_K_M quantization — 8 GB runs 7–8B models; a 70B model needs about 42 GB for its weights, and Llama 4 Scout (109B MoE) about 67 GB.
VRAM requirements at Q4_K_M quantization — 8 GB runs 7–8B models; a 70B model needs about 42 GB for its weights, and Llama 4 Scout (109B MoE) about 67 GB.
Local LLMs cost nothing per token after the hardware purchase; cloud APIs charge per token, with prices that vary widely by model and provider.
Local LLMs cost nothing per token after the hardware purchase; cloud APIs charge per token, with prices that vary widely by model and provider.
Open-weight local models by memory tier, 2026: Qwen3.8-27B (24 GB GPU), gpt-oss-20b (16 GB), Gemma 4 12B (12 GB), Qwen3.5 4B (8 GB RAM) and Phi-4 Mini (CPU-only).
Open-weight local models by memory tier, 2026: Qwen3.8-27B (24 GB GPU), gpt-oss-20b (16 GB), Gemma 4 12B (12 GB), Qwen3.5 4B (8 GB RAM) and Phi-4 Mini (CPU-only).
Ollama terminal: two commands download and run Llama 3.2 locally — typically under 10 minutes, depending on your connection.
Ollama terminal: two commands download and run Llama 3.2 locally — typically under 10 minutes, depending on your connection.
PromptQuorum dispatches one prompt simultaneously to your local Ollama model and 25+ cloud APIs — compare results side-by-side in one view.
PromptQuorum dispatches one prompt simultaneously to your local Ollama model and 25+ cloud APIs — compare results side-by-side in one view.

Compliance & Regional Context

EU / GDPR

Local LLMs process data on-premises. With full-disk encryption and access logging, on-premises inference keeps personal data inside your own systems. If no third party can access the data, no processor agreement under GDPR Article 28 is needed for the inference step; your other obligations (lawful basis, security, records of processing) still apply. Ollama binds to `localhost` by default — no external exposure.

Japan / APPI

Japan's Act on the Protection of Personal Information (APPI) restricts cross-border transfers of personal data. Running a model locally avoids a cross-border transfer for the inference step. METI's 2024 AI governance guidelines encourage privacy-preserving AI, and local deployment is consistent with that direction.

China / CAC

The Cyberspace Administration of China's Interim Measures for Generative AI Services (2023) require providers of public-facing generative AI services to register. A model that runs only on your own premises for internal use is generally not a public-facing service, but how the rules apply depends on the use case — check with counsel before relying on this.

Download Local LLMs Reference Card (PDF)

Improve Your Results

Running a local model? Your output quality depends on how you prompt it. Learn systematic techniques to get better answers from any local LLM.

Quick Answers, No Reading Required

Got a specific question — how much VRAM, which quantization, Ollama vs LM Studio? Prompt Bites answers 100+ of these in under a minute each.

PromptQuorum connects to your local LLM (Ollama, LM Studio, Jan AI) and dispatches your prompt to 25+ cloud models simultaneously — compare local vs cloud results in one view.

Try PromptQuorum free →

Discuss this page with an AI

  1. 1. Click your AI. The briefing is copied and the AI opens in a new tab.
  2. 2. In its chat box, paste with Ctrl+V (⌘+V on Mac) — or attach the downloaded file.
  3. 3. Send. The AI will ask what you want to discuss.

MCP