关键要点
- Mini PCs are compact desktops (5–10L volume) that fit under monitors or in media centers.
- RTX 4060 Ti (8GB) is the sweet spot: runs 7B models smoothly, fits in mini-ITX form factor, stays under $400.
- Intel NUC 13 Pro with RTX 4060 Ti docked externally: $1,200–1,400 total. Silent operation, small footprint.
- ASUS PN51 (mini-ITX barebone) + RTX 4060 Ti: $800–1,000. Best DIY mini-PC value.
- Expect 15–25 tokens/sec on 7B models in mini PCs (same as full-size rigs, just smaller case).
- Cooling is tight: GPU fans run loud at full load. Plan for 60–70°C GPU temps.
- Can't fit 70B models (need 24GB VRAM). Mini PCs max out at 13B comfortably.
- Great for living room, office corner, or silent workspace. Poor for enterprise (no expansion).
What Makes a Mini PC Suitable for Local LLMs?
A viable mini PC for local LLMs needs:
- PCIe x16 slot (full length): To fit a discrete GPU. Some mini PCs use mezzanine connectors or USB-C external docks; avoid these for LLMs.
- Power budget: Minimum 450W PSU. RTX 4060 Ti (160W) + CPU (65W) + motherboard (50W) = 275W, but spikes to 400W+.
- Cooling: Active case fans + heatsink. Passive cooling works for 7B at low batch sizes; sustained inference needs forced air.
- Enough storage: 1TB SSD for OS + 2–3 large models.
Best Mini PC Platforms
- Intel NUC 13 Pro (Core i7): Compact, upgradeable, runs 65W CPU. GPU via Thunderbolt 3 eGPU dock. $600 base + $400 RTX 4060 Ti + $200 dock = $1,200. Best build quality.
- ASUS PN51 or PN52 (mini-ITX): Barebone (no CPU/RAM/SSD). Add Ryzen 5 ($150) + 32GB RAM ($80) + 1TB SSD ($70) + RTX 4060 Ti ($400) = $800–900. Best value.
- Giada F350 or Zotac ZBOX Sphere (pre-built): Integrated GPU, no discrete card slot. Works for 3B–7B only. Skip for serious work.
- Custom mini-ITX build: Sff cases (Lian Li A4, Thorin 12V). Most flexible, toughest assembly. $1,000–1,400 total.
GPU Options for Mini PCs
Mini-ITX slot space limits GPU length to 220mm max.
- RTX 4060 Ti (8GB): Fits mini-ITX easily. Runs 7B models. $280–320 new, $180–220 used.
- RTX 4070 (12GB): Slightly longer (may not fit all cases). Overkill for 7B, perfect for 13B. $350–450.
- RTX 4060 (6GB): Fits, but VRAM is tight. 7B at Q4 leaves no headroom. Not recommended.
- RTX A4000 (16GB, enterprise): Used, $200–250. Professional cooling, excellent VRAM. Check case compatibility.
Cooling Challenges in Compact Cases
Mini PC cases restrict airflow. GPU fans can be loud.
- Thermals: Expect GPU 60–70°C, CPU 55–65°C under sustained inference. Not dangerous, but fans spin faster.
- Noise: RTX 4060 Ti fans at full load = 50–60 dB (vacuum cleaner level). Acceptable for office, annoying for bedroom.
- Undervolting: Drop GPU core voltage 50mV, reduce temps 5–10°C, lose 0–2% speed. Use MSI Afterburner (Windows) or GPU-tool (Linux).
- Silent operation: Swap GPU fans for Noctua or BeQuiet! ($50–80 extra). Reduces noise 10–15 dB.
Mini PC Limitations for Local LLMs
Mini PCs trade performance ceiling for compactness.
- Max VRAM: 8GB–16GB GPU only. Can't fit RTX 4090 (dual slot, huge cooler).
- Max model size: Comfortably 13B. 70B is impractical.
- Upgrade path: Limited. Swapping GPU might require case mod. RAM usually upgradeable.
- Multi-GPU: Impossible in mini-ITX. A second discrete GPU won't fit.
- Long-term: Mini PC cases designed for office use, not 24/7 inference. Plan for dust cleaning yearly.
Common Mini PC Mistakes
- Buying a mini PC with integrated GPU thinking it can run 7B models. Integrated GPUs are 10× slower.
- Choosing a mini PC with external TB3 eGPU dock, expecting full speed. eGPU loses 15–25% bandwidth.
- Assuming a mini PC case will fit a full-size ATX PSU. Mini PCs need specialized SFF PSUs.
FAQ
Can I run 13B models smoothly on a mini PC?
Yes, at Q4 quantization with RTX 4070 or RTX A4000 (12GB+). RTX 4060 Ti (8GB) is too tight for comfortable 13B inference.
Is Intel NUC with external RTX 4060 Ti docked good for local LLMs?
Yes. TB3 eGPU loses 15–20% bandwidth, so expect 12 tok/s instead of 15 on 7B. Still usable. Great for small spaces.
How loud is a mini PC running LLMs?
RTX 4060 Ti at full load = 50–60 dB (loud). Undervolting or replacing fans can drop to 40–45 dB (acceptable office level).
Can I fit an RTX 4090 in a mini PC?
No. 4090 is dual-slot and huge (280mm+). Even custom SFF cases max at 220mm GPU length.
Is a mini PC better than a laptop for local LLMs?
Mini PC: Better thermals, upgradeable, full-size components. Laptop: Portable, no setup needed. Mini PC wins for desktop use.
What's the total cost of a mini PC for 7B inference?
ASUS PN51 build: $800–900. Intel NUC 13 + RTX 4060 Ti dock: $1,200–1,400. Both good, PN51 is better value.
Sources
- Intel NUC specifications and TB3 eGPU compatibility matrix
- ASUS PN51 / PN52 official documentation and user benchmarks
- SFF PC community (smallformfactor.net): Mini-ITX case compatibility & thermal data