What is the Fastest Model for Your Hardware?
Match your hardware to the right model — the wrong choice leaves 4–10× speed on the table. All tiers below are CPU-only unless noted. Speeds are PromptQuorum estimates for dual-channel RAM, not lab measurements; single-channel RAM or an older DDR4 laptop lands lower.
Your Hardware | Recommended Model | Ollama Command | Expected Speed |
|---|---|---|---|
| 4 GB RAM, CPU only | Qwen3 1.7B Q4_K_M | ollama run qwen3:1.7b | 25–40 tok/s |
| 8 GB RAM, CPU only | Phi-4-mini 3.8B Q4_K_M | ollama run phi4-mini | 15–25 tok/s |
| 8 GB RAM + Intel Iris iGPU | Qwen3 4B Q4_K_M | ollama run qwen3:4b | 12–20 tok/s |
| 16 GB RAM, CPU only | Qwen3 8B Q4_K_M | ollama run qwen3:8b | 8–15 tok/s |
| 16 GB RAM + iGPU | Llama 3.2 3B Q4_K_M | ollama run llama3.2:3b | 20–35 tok/s |
Which Model Should You Use?
Match your situation to the right model — this is the single most important decision:
- 8 GB RAM laptop (no discrete GPU): Phi-4-mini (3.8B) at Q4_K_M — 15–25 tok/s, handles coding and reasoning. `ollama run phi4-mini`
- 4 GB RAM, very old PC: Qwen3 1.7B Q4_K_M — 25–40 tok/s, fastest response on minimal RAM. `ollama run qwen3:1.7b`
- 16 GB RAM, no GPU: Qwen3 8B Q4_K_M — 8–15 tok/s, strong quality. `ollama run qwen3:8b`
- 8 GB RAM + Intel Iris iGPU: Qwen3 4B — est. 12–20 tok/s if Ollama uses the iGPU (run `ollama ps` and read the PROCESSOR column). `ollama run qwen3:4b`
- Want long context: Qwen3 4B (256K context on Ollama) or Llama 3.2 3B (128K). Qwen3 1.7B and 8B are listed with 40K.
- Newer generation, smallest files: Qwen3.5 0.8B (1.2 GB on Ollama) is smaller than Qwen3 1.7B (1.4 GB), so it should run faster on the same CPU at lower quality. Qwen3.5 2B (2.7 GB) and 4B (3.3 GB) have larger files than Phi-4-mini (2.5 GB) and Qwen3 4B (2.5 GB), so expect them to be slower. `ollama run qwen3.5:0.8b`. PromptQuorum has not timed Qwen3.5 on this guide's hardware set.
- Also worth testing on 8 GB RAM: Gemma 4 E2B (Google, released April 2026). It has about 2.3B effective parameters, but the Ollama download starts at 4.6 GB because of its embedding tables, so it is not a 1.5–2 GB model. `ollama run gemma4:e2b`. See the CPU-only guide for estimated speeds by memory type.
- For per-RAM-tier picks and thermals, see how to run a local LLM on a laptop.
Which Local LLM Should You Run on Your Hardware?
All tiers below are CPU-only or iGPU. Choose the largest model that fits your RAM at Q4_K_M and leaves headroom for the operating system and context. Speeds are estimates.
Hardware | Model | Quant | Speed | Experience |
|---|---|---|---|---|
| 4 GB RAM, CPU only | Qwen3 1.7B | Q4_K_M | 25–40 t/s | fast, usable quality |
| 8 GB RAM, CPU only | Phi-4-mini 3.8B | Q4_K_M | 15–25 t/s | coding + reasoning |
| 8 GB RAM + Iris iGPU | Qwen3 4B | Q4_K_M | 12–20 t/s | if iGPU is used |
| 16 GB RAM, CPU only | Qwen3 8B | Q4_K_M | 8–15 t/s | strong quality |
| 16 GB RAM + iGPU | Llama 3.2 3B | Q4_K_M | 20–35 t/s | if iGPU is used |

GPU vs CPU for Local LLMs: Which Is Faster on Low-End Hardware?
Discrete GPU: typically 5–10× faster than CPU-only for the same model, because VRAM has far more memory bandwidth than system RAM. A used RTX 4060 8 GB (price varies) is a common budget upgrade. See budget GPU guide for cost-effective options.
iGPU (integrated): shares system RAM with the CPU, so the main limit — memory bandwidth — stays the same. Any gain is usually modest and depends on driver support. Ollama's documentation says Vulkan GPU support on Windows and Linux is enabled by default when the backend is installed, and lists AMD Ryzen AI iGPUs under ROCm; it does not list Intel integrated GPUs in a support table. Check `ollama ps`: the PROCESSOR column shows whether the model runs on CPU, GPU or a split.
CPU-only: runs on any PC with no driver setup. Speed is roughly memory bandwidth ÷ model file size, which is why dual-channel RAM and a small Q4_K_M file matter more than CPU clock speed.
Rule: Use a discrete GPU if you have one. Otherwise run the same prompt with the iGPU on and off (`OLLAMA_VULKAN=0` turns Vulkan GPUs off) and keep whichever is faster.

Why Smaller Models Are Faster on Low-End PCs
Model size directly determines speed. A 1B–3B model fits entirely in system RAM, allowing the CPU or GPU to stream data continuously. Larger models require memory swapping — moving data between RAM and disk — which slows generation dramatically (the bottleneck is disk I/O, not compute).
The hardware decision table above reflects this principle: the smallest files give the highest speed, while 13B+ models are impractical on low-end hardware because swapping dominates.
- 1B–3B models: Fit in 4–8 GB RAM → fastest generation → acceptable quality
- 7B models: Borderline on 8 GB systems → slower due to memory pressure → high quality
- 13B+ models: Need 16+ GB of RAM or swap heavily → too slow for interactive use on low-end PCs
How Fast Are Local LLMs on Low-End PCs?
On CPU-only systems, expect these estimated ranges:
- 3B models → 15–40 tokens/sec (older CPUs: 10–15, newer dual-channel DDR5 systems: 30–40)
- 7B–8B models → 8–15 tokens/sec (Q4_K_M on dual-channel RAM; single-channel RAM is lower)
- Estimate your own speed with one division. Generating each token requires reading the whole model from RAM once, so tokens per second can never exceed memory bandwidth ÷ model file size. Peak bandwidth = transfer rate (MT/s) × 8 bytes × channels.
- Worked examples (ceilings, not predictions): dual-channel DDR4-3200 gives 3200 × 8 × 2 = 51.2 GB/s; dividing by the 2.5 GB Phi-4-mini file gives about 20 tok/s at most. Dual-channel DDR5-5600 gives 89.6 GB/s, or about 36 tok/s at most for the same file. A single-channel DDR4-3200 laptop gives 25.6 GB/s, or about 10 tok/s at most. Real speed is lower than the ceiling.
- Intel N100 mini PCs have one memory channel (Intel specifications: DDR4-3200, DDR5-4800 or LPDDR5-4800). With DDR4-3200 the ceiling for the 1.4 GB Qwen3 1.7B file is about 18 tok/s; with DDR5-4800 about 27 tok/s. Expect the N100 to land below the dual-channel ranges in this guide.
- This is slower than cloud chatbots but sufficient for interactive use. A 3B model at 25 tok/sec generates a 500-token response in 20 seconds — acceptable for non-time-critical tasks like code review, summarization, and creative writing.
How Does Quantization Affect Speed on Low-End PCs?
Q4 (4-bit): the standard choice — roughly a quarter of the 16-bit file size with a small quality loss on most tasks. For details on all quantization levels and how they work, see the full guide.
Q3 (3-bit): a smaller file and a larger quality loss than Q4. Acceptable for chat when Q4 does not fit in RAM.
Q2 (2-bit): the smallest file and the largest quality loss. Risky; use only if Q4 and Q3 run out of memory.
Speed impact: because CPU decoding is limited by memory bandwidth, a smaller file generates faster in roughly the same proportion. Q3 and Q2 files are faster than Q4, but the quality drop is usually not worth it on models this small.
Strategy: pick the model size first, then use Q4_K_M. A small model at Q4_K_M usually beats a larger model squeezed to Q2, which loses accuracy and still needs more RAM.
Faster models trade quality for speed — but tuning temperature and top-p recovers much of that quality loss. Lower temperature (0.1–0.3) on fast models produces more consistent output than default settings. See temperature and top-p explained for the exact settings.
How Do You Speed Up CPU-Only Inference?
- Fill both memory channels: two matching RAM sticks double peak memory bandwidth versus one stick, which is the biggest single speed-up on a CPU-only PC. Single-stick laptops and mini PCs run at half the ceiling.
- Shrink the context window: the KV cache grows with context length. Set `OLLAMA_CONTEXT_LENGTH=2048` (the Ollama default is 4096), or `/set parameter num_ctx 2048` inside `ollama run`.
- Quantize the KV cache: `OLLAMA_FLASH_ATTENTION=1` with `OLLAMA_KV_CACHE_TYPE=q8_0` roughly halves KV-cache memory compared with the f16 default, which helps most on 4 GB and 8 GB machines.
- Keep the model loaded: Ollama unloads a model after 5 minutes by default. `OLLAMA_KEEP_ALIVE=30m` avoids reloading the file for every session.
- Close memory-hungry apps: browsers with many tabs push the model into swap, which makes generation many times slower.
- Test the iGPU: run `ollama ps` while a model is loaded. If PROCESSOR shows 100% CPU, compare against a GPU run; if the iGPU is unstable, set `OLLAMA_VULKAN=0`.
Can You Run Local RAG on 2 GB of RAM?
Yes, for small document sets: Qwen3 0.6B (523 MB on Ollama) for answers plus the nomic-embed-text embedding model (274 MB) for search need about 0.8 GB of model files in total. That fits when 2 GB of RAM is actually free, for example on a light Linux install, an old netbook or a PC with other apps closed. Expect simple answers drawn from your documents, not deep reasoning.
- Chat model: `ollama pull qwen3:0.6b` (523 MB, 40K context on Ollama). With a little more free RAM, `qwen3.5:0.8b` (1.2 GB) is the newer small option.
- Embedding model: `ollama pull nomic-embed-text` (274 MB, 2K context window). Split documents into chunks of a few hundred tokens so each chunk fits the 2K window.
- Retrieve few chunks: pass only the top 2–3 matching chunks to the chat model. Every extra chunk lengthens the context and grows the KV cache.
- Keep the context short: set `OLLAMA_CONTEXT_LENGTH=2048` so the KV cache stays small.
- Check the answers: a 0.6B model is good at pulling a fact out of a retrieved passage and weak at multi-step reasoning, so verify important answers against the source passage.
- When 2 GB is too tight: step up to Qwen3 1.7B (1.4 GB) on 4 GB of RAM, or Phi-4-mini on 8 GB, and keep the same embedding model.
How Fast Are These Models? Estimated Speeds
Estimated speeds by hardware class, not lab measurements. The ranges are PromptQuorum's own approximations, derived from each class's memory-bandwidth ceiling, for Ollama default settings and Q4_K_M files. Treat them as ceilings for a healthy install; your own `ollama run <model> --verbose` output reports the real tokens per second:
- Intel N100 mini PC (single-channel RAM, 4 GB tier) + Qwen3 1.7B Q4_K_M: up to about 18 tok/s with DDR4-3200 and about 27 tok/s with DDR5-4800 (bandwidth ceilings). `ollama run qwen3:1.7b`
- Modern dual-channel i5/Ryzen 5 (4 GB tier) + Qwen3 1.7B Q4_K_M: est. 25–40 tok/s. `ollama run qwen3:1.7b`
- Core i5-1235U class laptop (8 GB RAM) + Phi-4-mini Q4_K_M: est. 15–22 tok/s on LPDDR5; DDR4 laptops land lower. `ollama run phi4-mini`
- Ryzen 5 5600G class desktop (8 GB RAM) + Qwen3 4B Q4_K_M: est. 15–20 tok/s (dual-channel DDR4-3200 ceiling is about 20); an iGPU shares the same RAM bandwidth, so do not expect more.
- Intel Iris Xe class laptop (8 GB RAM) + Qwen3 4B Q4_K_M: est. 12–18 tok/s. `ollama run qwen3:4b`
- Ryzen 7 7700X class desktop (16 GB RAM, dual-channel DDR5) + Qwen3 8B Q4_K_M: est. 8–13 tok/s. `ollama run qwen3:8b`
- Iris Xe class laptop (16 GB RAM) + Llama 3.2 3B Q4_K_M: est. 20–30 tok/s. `ollama run llama3.2:3b`
What is Actually "Fast" for Local LLMs?
Speed feels different depending on the task — use this as your reference:
If your model runs below 15 tok/sec, downgrade model size (7B → 3B) or drop one quantization level (Q5 → Q4) before buying new hardware.
- Below 10 tok/sec → feels broken. Words appear one at a time with noticeable pauses. Unusable for interactive chat.
- 15–25 tok/sec → acceptable. Readable speed for most users. Good for Q&A, summaries, and coding help.
- 30+ tok/sec → smooth. Feels like a real assistant. Comfortable for all interactive tasks.
- 60+ tok/sec → instant. Faster than you can read. Ideal for real-time autocomplete and rapid iteration.
What to Avoid on Low-End PCs
- Do not run 13B+ models — they exceed RAM limits. A 13B model at Q4 needs roughly 8 GB for the file alone, which is beyond what an 8 GB PC can hold once the operating system is counted. Stick to 8B and below on 16 GB, and 4B and below on 8 GB.
- Avoid Q8 quantization — slower with minimal quality gain. Q8 roughly doubles the file size of Q4, and a bigger file means fewer tokens per second on a bandwidth-limited CPU. For 4 GB systems, Q8 is impractical; for 8 GB systems, Q4 remains optimal. Q3 is the only trade-off worth considering when Q4 runs out of memory.
- Do not expect real-time autocomplete performance. At 3 tok/sec on CPU, generating 50 tokens takes about 17 seconds. Interactive autocomplete needs far higher speeds. Local LLMs on low-end CPUs work for batch chat, drafting, and review — not live autocomplete or code-as-you-type scenarios.
- Do not use CPU-only inference for production chatbots. Acceptable for internal tools, prototypes, and offline batch work. A low-end CPU serves one user at a time at low speed, while a hosted API scales to many. Use local inference for privacy-critical or offline scenarios, not speed-critical ones.
Common Mistakes
- Mistake: Running one RAM stick. Problem: a single memory channel halves peak bandwidth, so the same model generates at roughly half the speed. Fix: install two matching sticks (or buy a dual-channel mini PC) before buying a faster CPU.
- Mistake: Judging speed by CPU model alone. Problem: a newer CPU on slow single-channel DDR4 can be slower than an older CPU on dual-channel DDR5, because token generation is limited by memory bandwidth. Fix: compare RAM speed and channels, then read the real figure from `ollama run <model> --verbose`.
- Mistake: Quantizing to Q2 to force 7B into 4GB. Problem: Q2 quantization often causes out-of-memory crashes due to KV cache overhead during inference, and the output quality drops sharply. Fix: Use a 3B model at Q4 instead.
- Mistake: Leaving the context window large. Problem: a long context grows the KV cache and pushes a small machine into swap. Fix: set `OLLAMA_CONTEXT_LENGTH=2048` unless you need long documents.
- Mistake: Using the wrong Ollama slug for your model. Problem: `ollama run phi` loads Phi-2, not Phi-4 Mini. Fix: Use `ollama run phi4-mini` for the current Phi mini model. Always check ollama.com/library for exact model tags.
Local LLMs on Low-End PCs: Regional Context
EU / GDPR: Local on low-end hardware: when run offline, no inference data leaves the device — for many SMEs and freelancers a technically straightforward way to avoid Art. 44 GDPR transfer risks. Whether the EU AI Act or other obligations apply to your use depends on who deploys the system and for what purpose; this guide is not legal advice. For German SMEs weighing local LLMs for internal business tasks, keeping documents on the device avoids sending them to a third-party processor, but overall data protection compliance still depends on your full operational setup, not the inference architecture alone.
Japan: Japan's METI AI guidelines for business emphasize responsible handling of data. CPU inference on low-end hardware, while slow, keeps prompts on the device — once the model is downloaded, inference itself needs no API calls. For Japanese-language tasks on Qwen3 at CPU speeds in the ranges above, a few seconds per sentence is acceptable for non-time-critical document summarization.
China: Local inference of Qwen3 and DeepSeek-R1 distills on consumer hardware is an option for users in China who cannot reach non-Chinese cloud APIs. Qwen3 0.6B, 1.7B and 4B run on CPU-only hardware, providing a functional alternative to cloud APIs for users with constrained hardware.
Common Questions About Running Local LLMs on Low-End PCs
What qualifies as a low-end PC for running local LLMs?
A low-end PC for local LLMs is any CPU-only machine (no discrete GPU) with 4–16 GB of system RAM. This includes most laptops with Intel Iris or AMD Radeon integrated graphics, older desktop PCs with no dedicated GPU, and mini PCs like the Intel N100. The key constraint is system RAM to hold model weights, not raw CPU clock speed.
Can I run a 7B-class model with no GPU at all?
Yes — Qwen3 8B Q4_K_M runs at an estimated 8–15 tok/s on 16 GB RAM, CPU-only (estimate for a dual-channel Ryzen 7 7700X-class desktop). It is slower than the smaller picks but delivers noticeably stronger quality. On 8 GB RAM, stick to Phi-4-mini (3.8B) instead — an 8B model at Q4_K_M needs headroom an 8 GB system does not reliably have.
Is CPU inference usable for chatbots?
Yes, for interactive use once you pick the right model size. Phi-4-mini at 15–25 tok/s (8 GB RAM) and Qwen3 1.7B at 25–40 tok/s (4 GB RAM) are both fast enough for real-time chat. A 7B-class model on CPU (8–15 tok/s) is better suited to batch tasks — drafting, summarization, offline review — than live back-and-forth conversation.
Should I use Phi-4-mini or Qwen3 1.7B on CPU-only hardware?
Use Phi-4-mini (3.8B) if you have 8 GB RAM — it handles coding and reasoning at 15–25 tok/s. Use Qwen3 1.7B if you have only 4 GB RAM or want the fastest possible response (25–40 tok/s) at the cost of some quality on complex tasks.
How do I check if I have any GPU at all?
Run `nvidia-smi` in terminal for NVIDIA discrete GPUs. If it returns "command not found," check `lspci` (Linux) or Device Manager (Windows) for an Intel Iris Xe or AMD Radeon integrated GPU — whether Ollama can use it depends on driver and backend support, so confirm with `ollama ps`.
How does quantization affect inference speed?
Quantization primarily reduces memory bandwidth requirements, not computation. A Q2 (2-bit) file is smaller than a Q4 (4-bit) file, so the CPU reads fewer bytes per token and generates faster, roughly in proportion to the file size. However, Q2 loses much more output quality. The practical rule: use Q4_K_M as default for every tier in this guide, drop to Q3 only if Q4_K_M does not fit in available RAM.
Can I use quantization below Q2?
Technically yes (1-bit IQ1 formats exist in llama.cpp), but output quality collapses at that size. Not recommended for any practical use case, including the 4 GB RAM tier covered in this guide — pick a smaller model at Q4_K_M instead.
Is CPU + iGPU hybrid inference supported?
Sometimes. Ollama can split a model between CPU and GPU when it finds a supported GPU backend, and `ollama ps` shows the split in its PROCESSOR column. An iGPU shares system RAM with the CPU, so memory bandwidth — the main speed limit — does not increase. On an 8 GB RAM system with an Intel Iris Xe iGPU, Qwen3 4B is estimated at 12–20 tok/s if the iGPU is used. Test with `OLLAMA_VULKAN=0` to compare against CPU-only.
What is the fastest local LLM for a low-end PC?
Qwen3 1.7B at Q4_K_M is the fastest of the tier picks in this guide (smaller models such as Qwen3 0.6B are faster still, at much lower quality) — an estimated 25–40 tok/s on a modern i5/Ryzen 5 with 4 GB RAM and no GPU. For better quality at a similar speed tier, Phi-4-mini (3.8B) runs 15–25 tok/s on 8 GB RAM and handles coding and reasoning noticeably better.
Can I run a local LLM on 4 GB RAM?
Yes — Qwen3 1.7B at Q4_K_M is the recommended pick for 4 GB RAM, CPU-only, reaching 25–40 tok/s on a modern i5/Ryzen 5. It is the fastest of the tier picks in this guide and fits comfortably with headroom for the OS.
Is a GPU required for speed?
No — every model in this guide runs CPU-only. An Intel Iris iGPU may add a modest speed boost (Qwen3 4B: est. 12–20 tok/s if the iGPU is used) but is optional and depends on driver support. A discrete GPU like a used RTX 4060 8 GB is typically 5–10× faster than CPU-only for the same model, but is a separate upgrade path, not a requirement.
Is an Intel Iris iGPU good enough for local LLMs?
It can help, but it is not guaranteed. On 8 GB RAM with an Intel Iris Xe iGPU, Qwen3 4B is estimated at 12–20 tok/s if Ollama actually uses the iGPU; on 16 GB RAM, Llama 3.2 3B is estimated at 20–35 tok/s. Because an iGPU shares system RAM with the CPU, the gain is usually modest. Run the same prompt with and without the iGPU and keep the faster setting.
Should I use Q2 or Q4 quantization on low-end hardware?
Start with Q4_K_M — it fits every model tier in this guide (1.7B through 8B) with a small quality loss. Only drop to Q3 or Q2 if Q4_K_M causes out-of-memory errors. Q2 files are faster because they are smaller, but the quality loss is noticeable on reasoning and coding tasks.
How do I check my available RAM on Windows?
Open Task Manager → Performance → Memory. The total shown is your system RAM — match it to the hardware decision table above (4 GB, 8 GB, or 16 GB tiers) to pick the right model.
Does Ollama automatically use my integrated GPU?
Only where a supported backend exists. Ollama's documentation says Vulkan GPU support on Windows and Linux is enabled by default when the backend is installed, and it lists AMD Ryzen AI iGPUs under ROCm; it does not list Intel integrated GPUs in a support table. Verify with `ollama ps` after loading a model — the PROCESSOR column shows 100% CPU, 100% GPU or a split. Set `OLLAMA_VULKAN=0` to turn Vulkan GPUs off.
What is the best free alternative to ChatGPT on a low-end PC?
Phi-4-mini (3.8B) at Q4_K_M on Ollama with 8 GB RAM, CPU-only, gives 15–25 tok/s — enough for chat, summaries, and coding help. Quality is lower than a frontier cloud model but sufficient for everyday tasks. Completely free, runs offline, no API key needed.
How much faster is a discrete GPU than CPU for local LLMs?
Typically 5–10× faster for the same model. A used RTX 4060 8 GB (price varies) usually outpaces the CPU-only setups covered in this guide. An Intel Iris iGPU sits in between at best — for example Qwen3 4B is estimated at 12–20 tok/s if the iGPU is used, versus an estimated 8–15 tok/s CPU-only for the closest comparable tier.
Is a local AI model the same thing as a local LLM?
For this guide, yes. "Local AI model" is the everyday phrasing for what the field calls a local LLM (large language model) — text-generating weights you download once and run on your own machine, offline. The distinction only matters when the AI model is not text-based: image generators like Stable Diffusion and speech models like Whisper also run locally, but they have different hardware profiles and none of the tok/s figures in this guide apply to them.
What is the lightest local LLM that still writes usable text?
Qwen3 1.7B at Q4_K_M is the lightest model in this guide worth using — 1.4 GB on disk in Ollama, runs in 4 GB RAM, an estimated 25–40 tok/s CPU-only. Smaller options exist (TinyLlama 1.1B, Qwen3 0.6B) and they are faster still, but output quality drops sharply enough that they are better treated as autocomplete than as a writing assistant. Below roughly 1B parameters, no lightweight model produces reliably coherent long-form text.
Is Gemma 4 E2B faster than Qwen3 1.7B on a low-end PC?
Probably not on 4 GB machines. Gemma 4 E2B (Google, released April 2026) has about 2.3B effective parameters, but the Ollama download starts at 4.6 GB because of its embedding tables, and on a bandwidth-limited CPU a bigger file means fewer tokens per second. It is a better fit for 8 GB or 16 GB RAM systems. The CPU-only guide estimates 8–14 tok/s for it on dual-channel RAM. Qwen3 1.7B (1.4 GB) remains this guide's speed pick for the lowest-power hardware; try `ollama run gemma4:e2b` for comparison on 8 GB or more.
Why is my local LLM so slow on a low-end PC?
The usual cause is memory bandwidth, not CPU speed: each token requires reading the whole model file from RAM, so speed is capped at memory bandwidth divided by file size. Check four things: a single RAM stick (one channel halves the ceiling), a model file too big for free RAM (swapping makes generation many times slower), a large context window (set `OLLAMA_CONTEXT_LENGTH=2048`), and whether the model runs on CPU or GPU (`ollama ps`). Then switch to a smaller model such as Qwen3 1.7B or Phi-4-mini at Q4_K_M.
How fast is an Intel N100 mini PC for local LLMs?
The N100 has a single memory channel (DDR4-3200, DDR5-4800 or LPDDR5-4800, per Intel). For the 1.4 GB Qwen3 1.7B Q4_K_M file the bandwidth ceiling is about 18 tok/s on DDR4-3200 and about 27 tok/s on DDR5-4800, and real speed is lower. Phi-4-mini (2.5 GB) has a ceiling near 10 tok/s on DDR4 and 15 tok/s on DDR5. Use Qwen3 1.7B or smaller on an N100.
Can I run local RAG on a laptop with 2 GB of RAM?
Yes, for small document sets, if about 2 GB of RAM is free. Use Qwen3 0.6B (523 MB) as the chat model and nomic-embed-text (274 MB, 2K context) for the embeddings, keep chunks to a few hundred tokens, pass only the top 2–3 chunks to the model and set `OLLAMA_CONTEXT_LENGTH=2048`. Expect simple answers drawn from your documents rather than deep reasoning, and verify important answers against the source passage.
Should I use Qwen3.5 instead of Qwen3 on a low-end PC?
For the fastest response, stay with Qwen3 1.7B for now. Ollama lists Qwen3.5 0.8B at 1.2 GB, 2B at 2.7 GB and 4B at 3.3 GB, against 1.4 GB for Qwen3 1.7B and 2.5 GB for Qwen3 4B. On a bandwidth-limited CPU a larger file generates fewer tokens per second, so only Qwen3.5 0.8B should be faster than Qwen3 1.7B, at lower quality. Qwen3.5 2B and 4B are worth testing if you value newer training over speed. PromptQuorum has not timed Qwen3.5 on this guide's hardware set.
What is the largest model I can run on CPU only?
The limit is free RAM for the file plus context. Ollama lists about 5.2 GB for Qwen3 8B, 9.3 GB for Qwen3 14B and 20 GB for Qwen3 32B, so a rough rule is 0.65 GB per billion parameters at Q4. A 32 GB PC can hold Qwen3 32B, but the bandwidth ceiling on dual-channel DDR5-5600 is about 4.5 tok/s (89.6 GB/s ÷ 20 GB), which makes it a batch tool. Mixture-of-experts models such as Qwen3 30B-A3B (19 GB) read only a few billion active parameters per token, so they should run faster than their file size suggests.
Sources
- Phi-4 Mini Model Card — Microsoft Research. Developer-reported benchmark results; released 2025.
- Ollama library: Qwen3 — file sizes and context lengths for qwen3:0.6b, 1.7b, 4b and 8b (checked 2026-10-23).
- Ollama library: Qwen3.5 — file sizes for qwen3.5:0.8b, 2b and 4b (checked 2026-10-23).
- Ollama library: Gemma 4 — gemma4:e2b download size and 128K context (checked 2026-10-23).
- Ollama library: Phi-4-mini — 2.5 GB, 128K context (checked 2026-10-23).
- Ollama library: nomic-embed-text — 274 MB embedding model, 2K context window (checked 2026-10-23).
- Ollama documentation: GPU support — Vulkan, ROCm and GPU selection variables including OLLAMA_VULKAN.
- Ollama documentation: FAQ — OLLAMA_CONTEXT_LENGTH, OLLAMA_KV_CACHE_TYPE, OLLAMA_FLASH_ATTENTION, OLLAMA_KEEP_ALIVE and `ollama ps`.
- Intel Processor N100 specifications — one memory channel; DDR4-3200, DDR5-4800, LPDDR5-4800.
- Gemma 4 announcement coverage — release date 2 April 2026, about 2.3B effective and 5.1B total parameters (third-party summary).
- llama.cpp build documentation — CPU backends and build options.
