Best Local LLMs by VRAM Tier 2026: Ollama, LM Studio & Hardware Guide
Local LLMs are large language models that run entirely on your own computer — no internet connection, no API fees, and no data leaving your machine. Compared with the OpenAI API you trade a little quality for privacy and zero running cost. If you are just starting, install Ollama and run Llama 3.2 3B or Qwen3.5 4B on 8 GB of RAM in under 10 minutes; the tier list below is a ranking of the best models, GPUs, and tools by VRAM — 4GB, 12GB, 24GB, 32GB and 48GB — for every budget.
Start here
The questions readers ask first, each with the guide to open. Pick the one that fits you.
Explore by topic
15 topics, 174 guides. One click opens a topic right where you are.
Getting Started 12 Guides
Zero-to-running in under 10 minutes. OS-specific installation guides, first-model walkthroughs, and a privacy-first setup checklist for beginners. Ollama installs with a single command on macOS, Windows, and Linux. For 8 GB RAM, start with Llama 3.2 3B (Q4, ~2 GB) using `ollama pull llama3.2:3b`.
Hardware & Performance 11 Guides
VRAM is the primary constraint for local LLMs. A 7B model at Q4_K_M needs about 4–5 GB; a 70B model needs about 42 GB. Guides cover GPU selection (RTX 4070 Ti to RTX 5090), Apple Silicon, budget builds, and VRAM calculation for any model. See also: Fastest Local LLMs for Low-End PCs for CPU-only, 4 GB, and 8 GB VRAM speed benchmarks.
GPU Buying Guides 9 Guides
GPU selection by budget and use case, cost per token, power efficiency, thermal design, second-hand marketplace comparisons, and warranty trade-offs. A 24 GB card such as the RTX 4090 suits 27B-class models; running a 70B model takes about 48 GB. Prices have moved with the 2026 memory shortage, so each guide lists current figures.
Mini PCs & AI Computers 13 Guides
Compact machines for local AI: mini PCs, AMD Strix Halo and NVIDIA DGX Spark workstations, Mac mini and ready-made AI boxes, compared on memory, speed and price.
PC Builds & Laptops 11 Guides
Complete system builds, laptops, workstations, storage and operating-system choices at multiple price points for serious local inference.
Apple Silicon 11 Guides
Mac-specific guides for M-series Apple Silicon — unified memory sizing, MLX vs Ollama vs llama.cpp benchmarks, Whisper transcription on Metal, and Mac Mini/MacBook builds for local inference. Apple Silicon shares unified memory between CPU and GPU, so a 32 GB M-series Mac can run 30B-parameter models without a discrete GPU. MLX is Apple's own inference framework and is typically faster than llama.cpp on the same chip.
Mobile & On-Device AI 12 Guides
Local LLMs on iPhone, iPad, Android and tablets: which apps and small models run offline, what phone chips can do, and how on-device AI differs from running models on a PC.
Cloud GPUs & Renting 4 Guides
When renting a cloud GPU beats buying hardware: provider comparison, the EU data-protection angle, China cloud options and the on-prem versus rental break-even for enterprises.
Models by Use Case 20 Guides
Model rankings, benchmark comparisons, and use-case picks by memory tier. Current picks: Qwen3.8-27B for 24 GB cards, Gemma 4 12B for 12 GB, gpt-oss-20b for 16 GB, Qwen3 8B or Qwen3.5 4B for 8 GB machines, and Phi-4 Mini for CPU-only setups. Benchmark scores are vendor-reported unless marked as measured.
Tools & Interfaces 25 Guides
Ollama and LM Studio run many open models on macOS, Windows, and Linux. Ollama is CLI-first with a REST API; LM Studio provides a graphical interface with a built-in model browser. Guides cover both tools plus vLLM, llama.cpp, Open WebUI, and IDE integrations.
Developer & MLOps Tools 10 Guides
The stack around local models in production: coding assistants, API gateways, vector databases, fine-tuning and agent frameworks, evaluation, observability and AI code review.
Advanced Techniques 13 Guides
Fine-tuning, RAG pipelines, quantization deep-dives, distillation, model merging, and prompt optimization for production use. LoRA and QLoRA cut fine-tuning memory needs substantially compared with full fine-tuning. Local RAG workflows keep sensitive data on-premises while maintaining search quality.
Enterprise 15 Guides
Multi-GPU setups, inference optimization, model serving frameworks (vLLM, TensorRT-LLM), monitoring and observability, cost audits, and regulatory compliance. Running models locally avoids sending data to a third-party API and can lower licensing costs compared with SaaS, depending on usage; compliance still depends on your whole setup.
Privacy & Business 4 Guides
On-premises deployment for privacy and compliance work (GDPR, HIPAA, APPI, CAC). Air-gapped setups and access logging. Local LLMs avoid API vendor lock-in, can reduce the compliance audit surface, and keep proprietary data off third-party servers; compliance itself depends on your whole setup.
Cost & Comparisons 5 Guides
Break-even analysis: local vs cloud vs subscription models. Hidden SaaS costs: overage fees, enterprise seats, audit logs. Whether local hardware pays for itself depends mainly on usage hours per month; the calculators let you enter your own numbers.
Practical answers
Four practical answers, calculated with a simple memory formula (estimates, not measurements). The full tables are at the bottom of this page.
In short: 8 GB of RAM is enough for a 7B model, all data stays on your machine, and there are no API costs once you own the hardware.
Recently updated
The six pages with the newest change.
Go deeper
The other topic worlds, each in its own color.
Overview as a presentation

The slide deck below covers hardware requirements (8 GB VRAM for 7B models, about 42 GB for a 70B model), top open-source models 2026, Ollama setup in 5 minutes, Q4_K_M quantization, regional compliance (GDPR, APPI), and key takeaways. Download the PDF as a Local LLMs quick-reference card.
Frequently Asked Questions
What is a local LLM?
A large language model (e.g., Qwen3.8, Gemma 4, gpt-oss, DeepSeek) that runs on your own hardware instead of a cloud API. Prompts and files stay on your machine, it works offline, has no usage limits, and costs nothing per token after the hardware purchase.
How much VRAM do I need for a local LLM?
8 GB of VRAM runs 7–8B models at Q4 quantization (about 5 GB of weights). 16 GB handles 13–14B models comfortably. A 70B model needs about 42 GB for its Q4 weights, so a 48 GB setup (for example two RTX 4090s) is the practical minimum. Apple Silicon unified memory counts as VRAM, minus the 8 GB macOS keeps for itself.
What is the difference between Ollama and LM Studio?
Ollama is a CLI tool that runs models via simple terminal commands and exposes an OpenAI-compatible API at `localhost:11434`. LM Studio provides a desktop GUI, model browser, and built-in chat interface. Both support the same models.
Can local LLMs match cloud models like GPT or Claude?
Open-weight models such as Qwen3.8-27B and Gemma 4 handle everyday coding, writing and summarizing on a single 24 GB GPU. Leading cloud models still have an edge on complex multi-step reasoning. Vendor-reported benchmark scores differ by test, so try your own prompts before choosing.
How do I fine-tune a local model?
Fine-tuning requires 500+ labeled training examples, the QLoRA framework (reduces VRAM requirement via 4-bit quantization), 24 GB+ VRAM (or a cloud GPU rental), and 1–4 hours of training time for a 7B model.
What is the minimum hardware to run a local LLM in 2026?
Minimum: 8 GB RAM and any modern CPU (runs 3B–7B models at roughly 2–5 tokens/sec). Recommended: a GPU with 8 GB+ VRAM (RTX 3060 or newer) for roughly 15–40 tokens/sec on 7–8B models.
Are local LLMs free to use?
Mostly yes. Ollama is open source and LM Studio is free to use but closed source. Most open-weight models (Qwen, Gemma, gpt-oss, Mistral, DeepSeek) are free to download, but licenses differ and Llama uses a community license, so check the terms before commercial use. The main cost is your hardware.
What is the best local LLM for coding in 2026?
Qwen3.8 27B is the top pick for a 24 GB GPU (18 GB, 256K context). Devstral Small 2 (24B, 15 GB) suits 16 GB machines, and Qwen3 8B suits 8 GB. Codestral 22B remains a common choice for IDE autocomplete, and Phi-4 Mini is the practical CPU-only choice. Benchmark scores are vendor-reported; see the coding guide for details.
Can I run a local LLM without a GPU?
Yes. Any modern CPU can run 3B–7B models at Q4 quantization using Ollama (CPU mode) or LM Studio. Typical speed is roughly 2–8 tokens/sec for 7B models on a laptop CPU and faster for 3–4B models, compared with 20–50 tokens/sec on an RTX 4060. 7B Q4 needs about 5 GB of RAM (not VRAM). For CPU-only setups, Phi-4 Mini (3.8B) and Llama 3.2 3B offer the best quality-to-speed ratio.
How do I update local LLM models when new versions are released?
Ollama: run `ollama pull <model-name>` again — it downloads only changed layers. LM Studio: open the model browser, find the updated version, and download it. Old GGUF files are not automatically removed — delete them manually from ~/.ollama/models (Ollama) or ~/Library/Application Support/LM Studio/models (macOS) to free disk space. New models usually appear in the Ollama library within days of an official release.
What are the best Ollama models in 2026?
Top picks in the Ollama library: Qwen3.8-27B (our pick for a 24 GB card, `ollama pull qwen3.8:27b`, 18 GB), gpt-oss:20b (small, 16 GB class), Gemma 4 (vision and tool calling, E2B to 31B; `ollama pull gemma4:12b` is about 8 GB), Qwen3 8B (`ollama pull qwen3:8b`, about 5 GB) for 8 GB machines, and DeepSeek-R1 distills for step-by-step reasoning. Run any model with `ollama run <name>` after pulling.
What is the best local LLM for an RTX 3060 12 GB VRAM?
The RTX 3060 12 GB is a good local LLM GPU. Best fits: Qwen3 14B (about 9 GB, 9–12 tokens/sec), Gemma 4 12B (about 8 GB, 11–14 tokens/sec), Qwen3 8B (16–20 tokens/sec) or DeepSeek-R1 7B (10–12 tokens/sec). Llama 4 Scout does not fit: it needs about 67 GB. The 12 GB of VRAM also puts you above the RTX 3060 Ti (8 GB) and opens up 13–14B-class models at full quality.
Ollama vs LM Studio vs Jan.ai: which should I use?
Use Ollama if you want a CLI tool with an OpenAI-compatible API at localhost:11434 — best for developers and automation. Use LM Studio if you want a desktop GUI, built-in model browser, and chat interface — best for beginners. Use Jan.ai if you want a privacy-focused chat app with a built-in model store. All three support the same GGUF models. Setup time: Ollama 2 min, LM Studio 5 min, Jan.ai 5 min.
What are the best budget GPUs for local LLMs in 2026?
According to our budget GPU guide: a used RTX 3060 12 GB (about $270–300) is the cheapest card worth buying, running Qwen3 14B at 9–12 tokens/sec and Qwen3 8B at 16–20 tokens/sec. New alternatives are the RTX 5060 Ti 16 GB (about $570–600) and the Intel Arc B580 12 GB (about $250–290). Prices rose in 2026 because of a memory shortage, so check the guide for current figures. Minimum recommended: 8 GB VRAM for useful 7B inference.
Is a local LLM better than ChatGPT?
For privacy and cost, yes. For raw output quality on the hardest tasks, usually no: leading cloud models still outperform locally runnable models on complex reasoning. For everyday tasks, a 27B-class local model on a 24 GB GPU is a capable alternative at no per-query cost.
How much RAM do I need to run a local LLM?
Minimum: 8 GB RAM to run a 7B model at Q4 quantization. Recommended: 16 GB for 13B models and about 48 GB for 70B models (about 42 GB of Q4 weights plus headroom). On a Mac, unified memory is shared with macOS, which needs at least 8 GB, so subtract 8 GB from the total — a Mac with 24 GB has about 16 GB usable and runs a 13B model well, and a 70B model at Q4 needs a 64 GB Mac. GPU VRAM is equivalent to RAM for GPU inference.
How do I run a local LLM?
Install Ollama (ollama.com), then run one command: `ollama run llama3.1:8b`. The model downloads automatically and you can start chatting in under 5 minutes. No API key, no account, no internet connection after the initial download.
What is the best free local LLM in 2026?
For a 24 GB GPU: Qwen3.8-27B (18 GB, `ollama pull qwen3.8:27b`). For 16 GB: gpt-oss:20b or Gemma 4 12B. For 8 GB: Qwen3 8B. For CPU-only setups: Phi-4 Mini. All are open-weight and free to download via `ollama pull`; check each license before commercial use.
Are local LLMs private?
Yes, for inference. When running with Ollama or LM Studio, prompts, documents and responses are processed on your machine and are not sent to a model provider's server. Update checks and optional telemetry are separate, so review each app's settings. Local processing suits confidential and personal data and simplifies GDPR work, but legal compliance still depends on your whole setup.
Key Takeaways
- 8 GB RAM is enough to run a 7B model locally (Ollama or LM Studio, under 10 min setup)
- 48 GB of VRAM runs a 70B model at Q4 (about 42 GB of weights); Llama 4 Scout (109B MoE) needs about 67 GB
- Q4 quantization cuts memory to roughly a quarter of FP16 with a small quality loss — a 7B model fits in 4–5 GB VRAM
- Open-weight models such as Qwen3.8, Gemma 4 and gpt-oss now cover most everyday coding and writing tasks; leading cloud models still lead on the hardest multi-step work
- Zero API costs after hardware purchase — no usage limits, no vendor lock-in
- Prompts and files stay on your machine, with no cloud storage — check each app's telemetry setting; local processing simplifies GDPR work but does not replace it
- LoRA fine-tuning requires 500+ labeled examples and 24 GB+ VRAM (or cloud GPU for training)
- Qwen Local Deployment Guide 2026 — one-command Ollama setup for Qwen 3.6 27B and Qwen3 8B, plus Coder and VL tiers
- Best GPU Under $500 for LLM Inference — Intel Arc B580 12 GB leads; a used RTX 3060 12 GB is the CUDA runner-up
- DeepSeek vs Qwen: Local Comparison 2026 — benchmark head-to-head
- Alibaba Cloud vs Tencent Cloud GPU 2026 — GPU cloud for the Chinese market
- Local LLM Cost Calculator: Build vs Rent 2026 — 3-year ROI calculator
Newest Models Added
| Model | Pull Command | Download size | Notes |
|---|---|---|---|
| Qwen3.8-27B | ollama pull qwen3.8:27b | 18 GB | Alibaba. 256K context, text and image input; our pick for 24 GB cards |
| Gemma 4 12B | ollama pull gemma4:12b | 8 GB | Google. 256K context, vision and tool calling; fits a 12 GB GPU |
| gpt-oss 20B | ollama pull gpt-oss:20b | 14 GB | OpenAI. 128K context, open weights; fits a 16 GB GPU |
| Qwen3.5 4B | ollama pull qwen3.5:4b | 4 GB | Alibaba. Text and image input; our small-model pick for 8 GB machines |
Best Local LLM by VRAM Tier
Already know your GPU's VRAM? Find your tier below for the best-fit model — for the full formula, quantization math, and MoE exceptions, see the complete VRAM guide.
| VRAM | Best-Fit Model (Q4) | Example GPUs |
|---|---|---|
| 4 GB | Llama 3.2 3B or Phi-4 Mini (~2–2.5 GB) | RTX 3050 4 GB, GTX 1650 |
| 8 GB | Llama 3.1 8B or Qwen3 8B (~4-5 GB) | RTX 4060 Ti 8 GB, RTX 3070 |
| 12 GB | Qwen3 14B (~8.4 GB), or 7-8B at Q5/Q8 for higher quality | RTX 3060 12 GB, RTX 4070 |
| 16 GB | Qwen3 14B at Q5/Q8, or 22B-class (~13 GB) | RTX 4060 Ti 16 GB, RTX 4080 |
| 24 GB | Qwen3.8-27B (~18 GB, our top pick for 24 GB cards) | RTX 4090, RTX 3090 |
| 32 GB | 22-32B at full precision, or one card in a dual-GPU 70B rig | RTX 5090 |
| 48 GB | 70B models at Q4 (~42 GB); Q5 (~50 GB) does not fit | 2x RTX 4090, RTX 6000 Ada |
Largest LLM You Can Run Locally
Memory sets the limit, not the software: at Q4 a model needs about 0.6 GB per billion parameters plus 1–2 GB of headroom. One 24 GB card runs a 27B-class model; 48 GB runs a 70B model.
| Model | Quantization | Weights | Fits |
|---|---|---|---|
| Qwen3.8-27B | Q4 | ~18 GB | 24 GB VRAM |
| 70B (Llama 3.3, Qwen) | Q4 | ~42 GB | 48 GB VRAM |
| 70B | Q5 | ~50 GB | Above a single 48 GB card |
| 70B | Q8 | ~74 GB | Multi-GPU setup |
Short on VRAM? See how to run 70B models on 24 GB VRAM and the full VRAM guide.
Best Quantization for 16 GB of VRAM or RAM
Quantized size is roughly parameters × 0.6 GB at Q4, × 0.7 at Q5 and × 1.05 at Q8, plus 1–2 GB for context. On 16 GB of VRAM, Q5 is a good default for 12–14B models and a 22B model fits only at Q4. With 16 GB of system RAM and no GPU, halve the budget: a 12B model at Q4 (about 7.2 GB) is the comfortable choice.
| Model size | Q4 | Q5 | Q8 | Fits 16 GB VRAM |
|---|---|---|---|---|
| 8B | ~4.8 GB | ~5.6 GB | ~8.4 GB | Q4, Q5, Q8 |
| 12B | ~7.2 GB | ~8.4 GB | ~12.6 GB | Q4, Q5, Q8 |
| 14B | ~8.4 GB | ~9.8 GB | ~14.7 GB | Q4, Q5 |
| 22B | ~13.2 GB | ~15.4 GB | ~23.1 GB | Q4 |
For the quality trade-off between bit levels, see LLM quantization explained.
Best Models for a Mid-Range PC in LM Studio
A mid-range PC means 12–16 GB of VRAM, for example an RTX 3060 12 GB or RTX 4060 Ti 16 GB. In LM Studio, search for the model, take the Q4 file if its size plus 1–2 GB fits your VRAM, and step up to Q5 when there is room.
| VRAM | Model to try | Approx. size |
|---|---|---|
| 12 GB | Qwen3 14B at Q4 (or a 12B-class model) | ~8.4 GB (12B: ~7.2 GB) |
| 16 GB | Qwen3 14B at Q5 | ~9.8 GB |
| 16 GB | 22B-class model at Q4 | ~13.2 GB |
Setup: how to install LM Studio. Not sure which app? Read Ollama vs LM Studio.
Local RAG on a Low-RAM Laptop
RAG needs two small models: an embedding model (nomic-embed-text, 137M parameters) and a chat model. On a CPU-only machine the chat model needs about twice its Q4 GPU size, so a 1B model takes about 1.2 GB and a 3B model about 3.6 GB. A 2 GB laptop is the floor, with almost no headroom for the operating system and your index; 8 GB is the practical starting point.
| System RAM | Chat model (Q4) | Memory (CPU-only) |
|---|---|---|
| 2 GB | Llama 3.2 1B + nomic-embed-text | ~1.2 GB for the chat model, very tight |
| 8 GB | Llama 3.2 3B + nomic-embed-text | ~3.6 GB |
| 16 GB | 7–8B model | ~8.4–9.6 GB |
Figures are estimates from the VRAM formula, not benchmarks. Full setup: local RAG guide and best local RAG tools.
Ollama vs LM Studio vs Jan.ai: Which Should You Use?
| Feature | Ollama | LM Studio | Jan.ai |
|---|---|---|---|
| Interface | Terminal (CLI) | Desktop GUI | Desktop GUI + chat |
| API endpoint | localhost:11434 | localhost:1234 | localhost:1337 |
| Model browser | CLI only | Built-in | Built-in |
| Best for | Developers, automation | Beginners, GUI users | Privacy-first chat |
| Setup time | 2 min | 5 min | 5 min |
Compliance & Regional Context
EU / GDPR
Local LLMs process data on-premises. With full-disk encryption and access logging, on-premises inference keeps personal data inside your own systems. If no third party can access the data, no processor agreement under GDPR Article 28 is needed for the inference step; your other obligations (lawful basis, security, records of processing) still apply. Ollama binds to `localhost` by default — no external exposure.
Japan / APPI
Japan's Act on the Protection of Personal Information (APPI) restricts cross-border transfers of personal data. Running a model locally avoids a cross-border transfer for the inference step. METI's 2024 AI governance guidelines encourage privacy-preserving AI, and local deployment is consistent with that direction.
China / CAC
The Cyberspace Administration of China's Interim Measures for Generative AI Services (2023) require providers of public-facing generative AI services to register. A model that runs only on your own premises for internal use is generally not a public-facing service, but how the rules apply depends on the use case — check with counsel before relying on this.
Download Local LLMs Reference Card (PDF)
Improve Your Results
Running a local model? Your output quality depends on how you prompt it. Learn systematic techniques to get better answers from any local LLM.
Quick Answers, No Reading Required
Got a specific question — how much VRAM, which quantization, Ollama vs LM Studio? Prompt Bites answers 100+ of these in under a minute each.
PromptQuorum connects to your local LLM (Ollama, LM Studio, Jan AI) and dispatches your prompt to 25+ cloud models simultaneously — compare local vs cloud results in one view.
Try PromptQuorum free →