Skip to main content
PromptQuorumBuilt for humans. Structured for AI.

Can You Run Local LLMs on a Radeon RX 6800M?

By · Founder of PromptQuorum · Discovery engine for open-weight & open-source AI

Key Takeaways

  • ✓The RX 6800M is a mobile RDNA 2 chip (Navi 22, 12 GB VRAM) — NOT the desktop RX 6800 (Navi 21, 16 GB VRAM)
  • ✓Vulkan is the dependable path on Windows and Linux; ROCm does not officially support the 6800M, and on Windows Ollama's ROCm list starts at the RX 7600
  • ✓Estimated speed: roughly 45–60 tok/s on an 8B Q4_K_M model and 25–35 tok/s on a 14B model — extrapolated from a same-die RX 6700 XT benchmark, not measured by PromptQuorum
  • ✓Run plugged in: laptop GPUs apply stricter power limits on battery, which lowers inference speed
Can You Run Local LLMs on a Radeon RX 6800M?

Quick Answer

Yes. The Radeon RX 6800M has 12 GB GDDR6 VRAM and runs local LLMs well through the Vulkan backend in Ollama or llama.cpp on both Windows and Linux. ROCm does not officially support this chip; on Linux it only works through an unofficial override. An 8B model at Q4_K_M is an estimated 45–60 tok/s, based on a same-die RX 6700 XT benchmark, not a PromptQuorum measurement.

  • ▸Windows and Linux: Vulkan is the reliable path (Ollama enables it by default)
  • ▸ROCm: the 6800M is not on AMD's or Ollama's support list; Linux needs HSA_OVERRIDE_GFX_VERSION=10.3.0 and may be unstable
  • ▸12 GB VRAM fits models up to 14B at Q4_K_M with a modest context window
Hardware-SpecificIntermediate

What the Radeon 6800M Can Actually Run

The Radeon RX 6800M is a mobile RDNA 2 chip (Navi 22, gfx1031) with 12 GB GDDR6 VRAM — not the desktop RX 6800, which uses the larger Navi 21 die with 16 GB. With 12 GB, the 6800M fits models up to 14B at Q4_K_M without layer offloading, the same capacity as a desktop RTX 3060 12 GB. Longer context windows add KV-cache memory on top of the model weights.

Single-user generation speed is mostly limited by memory bandwidth. The 6800M uses the same Navi 22 die and 12 GB memory layout as the desktop RX 6700 XT, which scores about 84 tok/s on Llama 2 7B Q4_0 with the Vulkan backend in the llama.cpp Vulkan scoreboard. Scaling that figure to larger model files gives the estimates below. Laptop power limits and cooling can place a 6800M below a desktop card.

These speeds are estimates derived from that public benchmark, not tests PromptQuorum ran on a 6800M. Run llama-bench on your own machine for your real numbers. For a CUDA comparison at 8 GB VRAM, see the AMD 5700X + RTX 3070 Ti rig comparison.

ModelVRAM (Q4_K_M)Estimated speed (Vulkan)
Llama 3.1 8B~5 GB~45–60 tok/s
Qwen3 8B~5 GB~45–60 tok/s
Phi-4 14B~9 GB~25–35 tok/s
Qwen3 14B~9 GB~25–35 tok/s

Setting Up Local LLMs on the 6800M

On Windows and Linux, install Ollama — it uses the Vulkan backend for AMD GPUs without ROCm. The Ollama GPU documentation states Vulkan is enabled by default when the backend is installed; on Linux it may need extra Vulkan packages. You can also use a prebuilt Vulkan binary of llama.cpp and load a GGUF with -ngl 99 to offload all layers to the GPU.

ROCm is the harder path. The RX 6800M is not on AMD's ROCm support table or Ollama's AMD list, and on Windows Ollama's ROCm list only covers the RX 7600 and newer. On Linux, community reports use HSA_OVERRIDE_GFX_VERSION=10.3.0 to make ROCm treat the chip as gfx1030. This is unofficial and may crash or fall back to CPU, so try Vulkan first.

Always run plugged in — laptop GPUs enforce stricter power limits on battery, which lowers inference speed. For the full GPU comparison across NVIDIA and AMD, see the best GPUs for local LLMs guide.

Test your setup: run ollama run llama3.1:8b, then check ollama ps — the PROCESSOR column should read 100% GPU. If it shows CPU, check the Ollama server log for GPU detection messages.

Radeon RX 6800M vs. Desktop RX 6800

Searches for "RX 6800" often land on the mobile chip, but the two cards differ. The desktop RX 6800 has 16 GB of VRAM and more memory bandwidth, so it fits larger models and generates faster. The llama.cpp Vulkan scoreboard lists about 101 tok/s for the desktop RX 6800 against about 84 tok/s for the RX 6700 XT (Llama 2 7B Q4_0, Vulkan).

The desktop RX 6800 appears on Ollama's Linux AMD support list, while the 6800M does not. Everything else on this page about Vulkan applies to both cards.

SpecRX 6800M (laptop)RX 6800 (desktop)
GPU chipNavi 22 (gfx1031)Navi 21 (gfx1030)
VRAM12 GB GDDR616 GB GDDR6
Memory bandwidth384 GB/s512 GB/s
Ollama Linux AMD support listNot listedListed
Ollama ROCm on WindowsNot listedNot listed

Related Guides

Quick Answers About Radeon 6800M and Local LLMs

Does the Radeon 6800M support ROCm officially?▾
No. The RX 6800M (gfx1031) is not on the AMD ROCm support table or on Ollama's AMD support list. On Linux it can sometimes run with HSA_OVERRIDE_GFX_VERSION=10.3.0, but that is an unofficial workaround. Vulkan is the supported path for this card.
How much VRAM does the Radeon RX 6800M have?▾
12 GB of GDDR6 on a 192-bit bus. That fits 8B models comfortably and 14B models at Q4_K_M with a modest context window. The 12 GB is dedicated GPU memory; system RAM does not add to it.
Does Ollama work on the RX 6800M?▾
Yes, through Vulkan. Ollama's documentation states Vulkan is enabled by default when the backend is installed. Ollama's ROCm path does not list the 6800M, so expect Vulkan rather than ROCm unless you use the unofficial Linux override.
Is the desktop RX 6800 better for local LLMs than the 6800M?▾
Yes, in capacity and bandwidth: 16 GB versus 12 GB of VRAM and about 512 GB/s versus 384 GB/s. It also appears on Ollama's Linux AMD support list. See the comparison table above.
Is the 6800M faster than RTX 3070 Mobile for LLMs?▾
The 6800M's 12 GB of VRAM versus 8 GB on most RTX 3070 Mobile configurations matters more for model fit than raw speed. At equal model size, the RTX 3070 Mobile benefits from mature CUDA support. Compare real results with llama-bench on your own machine.
How hot does the 6800M get running LLMs continuously?▾
Sustained inference loads the GPU much like a gaming session, so laptop temperatures can reach the 80s °C depending on cooling. Monitor your own temperatures, keep the vents clear, and consider a modest undervolt in Radeon Software (Windows) or CoreCtrl (Linux) if the laptop throttles. Undervolting is at your own risk: it can cause instability and may affect your warranty.
MCP