Can You Run a Local LLM on a Raspberry Pi 5?
This page contains links to third-party products for reference. PromptQuorum is not enrolled in any affiliate program β these are plain links that earn no commission. Clicking links and your next steps are entirely your own responsibility. These links do not represent any endorsement or verification by PromptQuorum.
Quick Answer
Yes, but only tiny models β a Raspberry Pi 5 (8GB) runs 1B-3B models at Q4 via llama.cpp on its CPU, at roughly 2-5 tokens/sec. There is no GPU acceleration available.
- βΈBuy the 8 GB Raspberry Pi 5 β the 4 GB model leaves too little headroom even for a 1B model plus the OS.
- βΈRealistic model range: Llama 3.2 1B/3B or Qwen3 0.6B at Q4 β anything larger is impractically slow.
- βΈNo GPU acceleration: llama.cpp does not support the Pi 5's VideoCore VII GPU β everything runs on the CPU.
Updated: 2026-07
Key Takeaways
- βYes, but only 1B-3B models at Q4 β anything larger is impractically slow on Pi 5 hardware
- βBuy the 8 GB configuration β the 4 GB model leaves too little headroom for model plus OS
- βExpect roughly 2-5 tokens/sec on CPU β no GPU acceleration path exists via llama.cpp
- βA Pi 5 is a fun learning project, not a practical daily-driver local LLM machine
The Honest Answer: Small Models Only, and Slowly
A Raspberry Pi 5 can run a local LLM, but only within the 1B-3B parameter range at Q4 quantization β models like Llama 3.2 1B, Llama 3.2 3B, or Qwen3 0.6B β at roughly 2-5 tokens per second through llama.cpp. That speed is usable for casual experimentation but noticeably slower than typing speed for longer responses, and far behind any GPU-equipped machine.
Buy the 8 GB configuration, not the 4 GB one. Even a 1B model plus Raspberry Pi OS overhead leaves little headroom on 4 GB, and you'll want the extra memory for a comfortable context window and any other software running alongside the model.
There is no GPU acceleration path available: the Pi 5's integrated VideoCore VII GPU is not supported by llama.cpp's inference backends (no Vulkan or OpenCL path currently targets it for LLM inference), so every token is computed on the quad-core ARM CPU. Treat a Pi 5 local LLM setup as an educational project or a very lightweight assistant β not a replacement for any GPU-equipped hardware, even a budget one.
Raspberry Pi 5 vs the Cheapest GPU Alternative
Even the least expensive GPU covered in the sub-$300 GPU guide runs 7B models at 15-20 tokens/sec β several times faster than a Pi 5's best case on models a fraction of the size. The Pi 5's appeal is its small footprint, low power draw, and novelty, not raw capability.
If your goal is a genuinely useful always-on local LLM assistant rather than a hobby project, a used GPU or a mini PC is the more practical investment β see the sub-$300 GPU guide or the always-on Ollama server mini PC guide.
Related Reading
- βΈBest Local LLM for 6 GB VRAM β a genuinely practical low-budget GPU alternative
- βΈHow Much RAM Does a 7B Model Need? β why 7B is out of reach for a Pi 5
- βΈBest Mini PC for an Always-On Ollama Server β a more practical always-on alternative
Frequently Asked Questions
Does the Raspberry Pi 5 need active cooling for LLM inference?βΎ
Can I use an AI accelerator hat to speed up the Pi 5?βΎ
Is a Raspberry Pi 5 good for running a voice assistant with a local LLM?βΎ
What is the minimum RAM for any local LLM on a Pi 5?βΎ
Related Prompt Bites