TL;DR
- 16GB: 7B models only (tight)
- 36GB: 13B comfortably, 34B Q4 tight
- 64GB: 34B Q5 comfortably
- 128GB: 70B Q5 comfortably
- Cannot upgrade after purchase — buy maximum at purchase time
Key Takeaways
- Unified memory = shared between CPU and GPU — all of it available to LLM models.
- RTX 4070 has 12GB VRAM + 32GB RAM (separate). Mac has unified = all available.
- 64GB Mac has ~56–60GB for LLMs after macOS overhead (4–8GB).
- Swap exists: macOS uses SSD if model exceeds free memory. Works but 5–10× slower.
- Model size in GB varies by quantization: Llama 3.1 8B is 16GB FP16, 5GB Q4, 8.5GB Q8.
- Rule: Buy maximum memory — cannot upgrade after purchase. Memory cost at sale is 5–10%; full Mac replacement later is 100%.
How Unified Memory Works for LLMs
Unified memory is shared between CPU and GPU — all of it available to the model. Unlike discrete GPU (RTX 4070 has 12GB VRAM + 32GB RAM separate), Apple Silicon shares one pool. 64GB Mac = 64GB available to model. macOS and apps use 4–8GB, leaving 56–60GB for LLM.
The Master Table: Memory Tier vs Model Size
| Model | Parameters | Q3_K | Q4_K_M | Q5_K_M | Q8 | FP16 |
|---|---|---|---|---|---|---|
| Phi-4 | 3.8B | 2.1 GB | 2.5 GB | 2.9 GB | 4.0 GB | 7.6 GB |
| Mistral 7B | 7B | 3.8 GB | 4.5 GB | 5.2 GB | 7.5 GB | 14 GB |
| Llama 3.1 8B | 8B | 4.2 GB | 5.0 GB | 5.8 GB | 8.5 GB | 16 GB |
| Llama 3.1 13B | 13B | 7.0 GB | 8.5 GB | 9.8 GB | 14 GB | 26 GB |
| Qwen2.5 34B | 34B | 17 GB | 20 GB | 24 GB | 36 GB | 68 GB |
| Llama 3.1 70B | 70B | 36 GB | 42 GB | 49 GB | 74 GB | 140 GB |
| Llama 3.1 405B | 405B | 200+ GB | 240 GB | 280 GB | 410 GB | 810 GB |
Add 4-8 GB for macOS overhead when calculating fit on your Mac.
Fits / Doesn't Fit Matrix
| Model + Quant | 16GB | 36GB | 64GB | 128GB |
|---|---|---|---|---|
| Phi-4 Q4 (2.5 GB) | ✓ Plenty | ✓ Plenty | ✓ Plenty | ✓ Plenty |
| Llama 3.1 8B Q4 (5 GB) | ⚠️ Tight | ✓ Comfortable | ✓ Plenty | ✓ Plenty |
| Llama 3.1 8B Q8 (8.5 GB) | ✗ Won't fit | ✓ Comfortable | ✓ Plenty | ✓ Plenty |
| Llama 3.1 13B Q4 (8.5 GB) | ✗ Won't fit | ✓ Comfortable | ✓ Plenty | ✓ Plenty |
| Qwen2.5 34B Q4 (20 GB) | ✗ Won't fit | ⚠️ Tight | ✓ Comfortable | ✓ Plenty |
| Qwen2.5 34B Q5 (24 GB) | ✗ Won't fit | ✗ Won't fit | ✓ Comfortable | ✓ Plenty |
| Llama 3.1 70B Q3 (36 GB) | ✗ Won't fit | ✗ Won't fit | ⚠️ Tight | ✓ Comfortable |
| Llama 3.1 70B Q4 (42 GB) | ✗ Won't fit | ✗ Won't fit | ⚠️ Very tight | ✓ Comfortable |
| Llama 3.1 70B Q5 (49 GB) | ✗ Won't fit | ✗ Won't fit | ✗ Won't fit | ✓ Comfortable |
| Llama 3.1 70B Q8 (74 GB) | ✗ Won't fit | ✗ Won't fit | ✗ Won't fit | ✓ Fits |
✓ Plenty = 4+ GB free | ✓ Comfortable = 2-4 GB free | ⚠️ Tight = under 2 GB free | ✗ Won't fit = uses swap or crashes
What Fits in Each Memory Tier (Practical)
- 116 GB (M5 base, MacBook Air)
Why it matters: Llama 3.1 8B Q4 fits (5GB model + 8GB OS = 13GB) ✓ but tight. Llama 8B Q8 won't fit without swapping. Whisper small fits alongside. - 236 GB (M5 Pro base)
Why it matters: Llama 3.1 8B Q8 fits comfortably. Llama 13B Q4 fits. Qwen2.5 34B Q4 barely fits (20GB + 8GB OS = 28GB). Multi-model: Whisper + LLaVA + TTS fit ✓ - 364 GB (M5 Pro max)
Why it matters: Qwen2.5 34B Q5 fits comfortably (24GB). Llama 70B Q3 barely fits. Multi-model stacks have plenty of room. - 4128 GB (M5 Max)
Why it matters: Llama 3.1 70B Q5 fits comfortably (49GB). 70B Q8 fits (74GB). Multi-modal: Whisper + 90B vision model + 8B LLM fit simultaneously ✓
Multi-Model Stack Memory Requirements
| Stack Use Case | Memory Needed |
|---|---|
| LLM only (Llama 8B Q4) | 5 GB + OS = 13 GB |
| LLM + STT (Llama 8B + Whisper large-v3) | 8 GB + OS = 16 GB |
| LLM + STT + TTS (voice assistant) | 9 GB + OS = 17 GB |
| LLM + Vision (Llama 8B + LLaVA 7B) | 11 GB + OS = 19 GB |
| Full multimodal (LLM + Vision + STT + TTS) | 14 GB + OS = 22 GB |
| LLM + RAG (Llama 8B + embeddings + ChromaDB) | 8 GB + OS = 16 GB |
| Heavy multimodal (Llama 70B Q4 + Vision 90B) | 100+ GB |
Stacks above 22 GB need 36GB minimum Mac. Stacks above 50 GB need 64GB minimum Mac. The heavy multimodal stack only works on 128GB M5 Max.
Context Window Adds Memory Overhead
KV cache scales with context length — the longer the context window, the more memory your model uses at runtime. This is a common gotcha that can push a tight setup into swap.
- Llama 3.1 8B at 8K context: +0.5 GB
- Llama 3.1 8B at 32K context: +2 GB
- Llama 3.1 8B at 128K context: +8 GB
- Llama 3.1 70B at 32K context: +6 GB
- Llama 3.1 70B at 128K context: +24 GB
Buy Maximum Memory — Here's Why
- Cannot upgrade Apple Silicon memory after purchase.
- Model sizes are growing: 8B today → 13–34B sweet spot in 2027.
- 16GB is already marginal for LLMs — 36GB minimum recommended.
- Price difference: 36GB→64GB costs ~$200 at purchase, saves buying new Mac in 2 years when models outgrow 36GB.
- Example: M5 Pro 36GB today costs $999; 64GB costs $1,199. Buying new Mac in 2 years: $1,500+ for same M5 Pro 64GB config (if available).
Quantization Impact on Quality
Q4_K_M (4-bit): ~1–2% quality loss vs FP16. Imperceptible for most uses. Best default.
Q5_K_M (5-bit): ~0.5–1% quality loss. Negligible. Recommended if you have spare memory.
Q8 (8-bit): ~0.1% quality loss. Essentially lossless.
Q3_K (3-bit): 3–5% quality loss. Noticeable on complex reasoning. Acceptable only for space-constrained scenarios.
Should I get 36GB or 64GB?
Get 64GB if budget allows ($200 more). 36GB works today but will feel tight in 12 months as models grow. 64GB is future-proof through 2027–2028.
Can I upgrade memory later?
No. Apple Silicon memory is soldered and non-upgradeable. Buy maximum at purchase time.
Why is 16GB not enough?
16GB for LLM + 4–8GB for macOS = 8–12GB available. Llama 8B Q4 needs 5GB, leaving no room for Whisper or other tasks. Too tight.
Do I really need 128GB?
Only if you regularly run 70B models or need simultaneous vision + LLM + STT. Otherwise, 64GB is plenty.
Is 48GB enough for local LLMs?
Yes — 48GB (available on M4 Pro and some M5 Pro configs) is a comfortable middle ground. Runs all 34B models, 70B Q3 at the edge, and full multimodal stacks. Better than 36GB, but if you can afford 64GB, the future-proofing is worth it.
How much memory for running Llama 3.1 70B locally?
Minimum 48GB (Q3 quantization, noticeable quality loss). Recommended 64GB (Q4 quantization, tight fit). Comfortable 128GB (Q5/Q8 quantization, high quality). The 64GB tier requires careful memory management; 128GB is the only worry-free option for 70B.
Do I need 128GB for local AI in 2026?
Only if you're running 70B models regularly or need simultaneous vision + LLM + STT stacks. For everyday LLM use (8B-34B models, RAG, coding assistance), 64GB M5 Pro is the sweet spot. 128GB is a 2-3× price jump for marginal benefit unless you specifically need 70B.