Whisper.cpp vs faster-whisper 2026: Local STT Benchmarks, Setup & GPU Acceleration

Quick Facts

• Both tools: Based on OpenAI's open-source Whisper model (MIT license). Same accuracy — different runtimes.
• whisper.cpp: Written in C/C++ by ggerganov. Supports CPU (AVX2/NEON), CUDA, Metal (Apple), Vulkan. No Python required.
• faster-whisper: Python library using CTranslate2. Supports CPU (int8) and CUDA. No Apple Metal support.
• Whisper model sizes: tiny (39M), base (74M), small (244M), medium (769M), large-v3 (1.55B). All use the same ggml / CTranslate2 format.
• Best model for most use cases: Whisper small — 3.4% WER, runs in 2 GB RAM, 6× real-time on modern CPU.
• RTX 4070 benchmark (large-v3): faster-whisper ~12× real-time; whisper.cpp CUDA ~8× real-time. faster-whisper wins on NVIDIA.
• M5 Pro benchmark (large-v3): whisper.cpp Metal ~10× real-time; faster-whisper CPU-only ~3× real-time. whisper.cpp wins on Apple.

Why Local Speech-to-Text?

• Privacy: Audio never leaves your machine, so no cloud processor is involved in transcription. Data-protection duties such as legal basis and secure storage still apply to you.
• Cost: Zero per-minute fees. A developer transcribing 8 hours of meetings per week saves $120–480/month at cloud STT pricing.
• Offline: Works on planes, in secure facilities, in areas without reliable internet. No API key management.
• Latency: No upload/download round-trip. For real-time voice interfaces, local processing reduces STT latency from 300–800 ms (cloud) to 50–300 ms.
• Customization: Fine-tune on domain-specific vocabulary. Run any model size that fits your hardware.
• Home assistant integration: Whisper running locally means wake words and voice commands never leave your home network. See local Whisper in Home Assistant → for the add-on setup that replaces cloud STT entirely.

Whisper Model Sizes — Foundation for Both Tools

Distil-Whisper: The Faster Alternative

• Parameters: ~756M — roughly half of large-v3's 1.55B, fitting in ~4 GB VRAM instead of ~10 GB.
• Speed: ~6× real-time on RTX 4070 (vs. 1× baseline for large-v3) — comparable to the medium model in speed, with large-v3-level accuracy.
• WER: ~2.6% on English — only ~0.1% higher than large-v3's 2.5%. In practice, the difference is inaudible on typical speech.
• Compatibility: Works with faster-whisper natively (WhisperModel("distil-large-v3", device="cuda", compute_type="int8")). For whisper.cpp, convert to GGML format using the distil-whisper GGML conversion script.
• Best for: Batch transcription jobs, server deployments with limited VRAM, and any use case where you want large-v3 quality at medium-model speed.
• Not for: Multilingual transcription — distil-large-v3 is English-only. For other languages, use large-v3 or medium.

whisper.cpp — The C/C++ Port

• Platform support: CPU (AVX2, AVX512, ARM NEON), Apple Metal (Core ML), CUDA (NVIDIA), Vulkan (AMD/Intel GPU), OpenCL.
• Apple Silicon advantage: whisper.cpp exports models to Core ML format, enabling inference on the Apple Neural Engine. Large-v3 runs at ~10× real-time on M5 Pro via Metal — faster than any cloud round-trip.
• Installation: Clone the repo, run make (or cmake). Pre-built binaries available for common platforms. No Python dependency.
• Model download: bash ./models/download-ggml-model.sh base.en — downloads the GGML-format model file (~142 MB for base).
• CLI example: ./main -m models/ggml-base.bin -f audio.wav — transcribes a WAV file to stdout. Add -l de for German.
• Real-time stream mode: ./stream -m models/ggml-base.bin --step 3000 --length 10000 — transcribes from microphone in 3-second chunks.
• Python wrapper: pywhispercpp provides a Python binding for whisper.cpp, enabling use in Python pipelines without sacrificing Metal acceleration.
• Limitation: No native VAD (voice activity detection). Stream mode requires tuning --step and --length parameters for your use case.

# Build from source (macOS / Linux)
git clone https://github.com/ggerganov/whisper.cpp
cd whisper.cpp
make -j4

# Download a model
bash ./models/download-ggml-model.sh large-v3

# Transcribe a file
./main -m models/ggml-large-v3.bin -f recording.wav

# Enable Metal on Apple Silicon (Core ML)
make -j4 WHISPER_COREML=1
./main -m models/ggml-large-v3-encoder.mlmodelc -f recording.wav

whisper.cpp Latest Version & Updates

• whisper.cpp v1.9.5 (October 6, 2026): Syncs ggml to v0.26.0 with CUDA, Vulkan, Metal, SYCL, OpenCL, WebGPU and CPU kernel updates, adds an abort callback during language detection, fixes an integer overflow in parallel chunk offsets, and makes the CLI return the correct error code when a file fails.
• whisper.cpp v1.9.4 (September 11, 2026): The server now returns the detected language, the encoder callback runs before language auto-detection, and Windows on ARM was added to the release job.
• whisper.cpp v1.9.3 (August 20, 2026): Syncs ggml to v0.20.2 with backend-wide performance improvements, Metal TQ2_0 quantization support, fused CUDA/SYCL kernels, and ARM/WASI fixes.
• whisper.cpp v1.9.0–v1.9.2 (June–August 2026): Point releases continuing the same ggml-sync and cross-backend performance line ahead of v1.9.3.
• whisper.cpp v1.8.6 (June 2, 2026): Reimplemented ffmpeg-transcode for clearer optional FFmpeg decoding of compressed audio (MP3, M4A) plus a CI examples-path fix.
• whisper.cpp v1.8.5 (May 2026): Streaming VAD (voice activity detection) improvements, server parameter-leak and token-timestamp fixes, and memory-leak fixes in the Ruby and VAD bindings.
• whisper.cpp v1.8.4 (March 19, 2026): Latest ggml sync with broad performance gains, a new -g / --gpu-device flag (and GGML_CUDA device selection) to pick a GPU, and UTF-8 segment-wrapping fixes.
• Native VAD is now built into whisper.cpp. Recent releases added a native voice-activity-detection path, narrowing a long-standing faster-whisper advantage — though faster-whisper's Silero VAD integration remains more turnkey.
• faster-whisper v1.2.1 (October 31, 2025): Latest tagged release; pip install faster-whisper pulls it. There is no tagged release in 2026, but the main branch is active: commits from October 16, 2026 add input validation, such as an error for languages a model does not support. v1.2.0 (August 2025) added distil-large-v3.5 support.
• CTranslate2 v4.8.3 (October 13, 2026): The inference engine under faster-whisper keeps shipping. The v4.8 line added Windows ROCm fixes, load-time hardening, and a fix for a division by zero in Whisper align() when a window has no frames.
• No new accuracy-class Whisper model in 2026. OpenAI's newest multilingual weights remain large-v3-turbo (October 2024; 809M parameters, decoder cut from 32 to 4 layers, MIT license), which trades a small quality loss for much higher speed and runs in both tools. distil-large-v3 stays the English-only speed option. Updates to either runtime are runtime and tooling changes, not accuracy changes.

faster-whisper — The CTranslate2 Port

• Platform support: CPU (int8 quantization) and NVIDIA CUDA GPU. No Apple Metal support — runs CPU-only on Mac.
• int8 advantage: CTranslate2 int8 quantization reduces VRAM by ~40% and increases inference speed by ~2× vs float16, with negligible WER impact (< 0.1% absolute).
• Installation: pip install faster-whisper — no compilation required (Python 3.9+). GPU use needs NVIDIA cuBLAS for CUDA 12 and cuDNN 9; older CUDA 11 / cuDNN 8 setups must pin an older ctranslate2 (3.24.0).
• Built-in VAD: faster-whisper includes Silero VAD integration, which automatically skips silent audio segments — critical for real-time transcription pipelines.
• Python-native: Direct Python API makes it trivial to chain with LLMs, audio processing libraries, and web frameworks.
• Speed: large-v3 int8 on RTX 4070 runs at ~12× real-time and uses ~2.5 GB VRAM. CPU int8 achieves ~20× real-time for the tiny model.
• Batch processing: faster-whisper supports batched inference for processing large audio archives efficiently.
• Limitation: No Metal support on Mac — runs CPU-only on Apple Silicon, achieving ~3× real-time for large-v3 vs. whisper.cpp's ~10× with Metal.

from faster_whisper import WhisperModel

# Load model (downloads automatically on first run)
model = WhisperModel("large-v3", device="cuda", compute_type="int8")

# Transcribe
segments, info = model.transcribe("audio.wav", beam_size=5)

print(f"Detected language: {info.language} (probability: {info.language_probability:.2f})")
for segment in segments:
    print(f"[{segment.start:.2f}s → {segment.end:.2f}s] {segment.text}")

Get ggml-base.bin via pip (Python Setup)

• Install the binding: pip install pywhispercpp. For NVIDIA CUDA acceleration, build with GGML_CUDA=1 pip install git+https://github.com/absadiki/pywhispercpp.
• Auto-download ggml-base.bin: from pywhispercpp.model import Model then model = Model("base") downloads ggml-base.bin from Hugging Face (ggerganov/whisper.cpp) on first run and caches it. Use "base.en" for the English-only variant.
• Manual download (CLI route, no pip): From a cloned whisper.cpp repo, bash ./models/download-ggml-model.sh base saves the file to models/ggml-base.bin — the same file pywhispercpp fetches.
• Direct download: curl -L -o ggml-base.bin https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-base.bin pulls the model with no script. Swap base for tiny, small, medium, or large-v3.
• Do not confuse the packages: pip install openai-whisper installs OpenAI's original PyTorch Whisper (it uses .pt checkpoints, not ggml-base.bin). pip install faster-whisper uses the CTranslate2 format. Only pywhispercpp consumes GGML ggml-*.bin files.
• Where the file lives: pywhispercpp stores downloaded GGML models under its module cache (or a path you pass via Model("base", models_dir="...")), so the same ggml-base.bin is reused across runs — no re-download.

# 1. Install the pip binding for whisper.cpp
pip install pywhispercpp

# 2. ggml-base.bin downloads automatically on first use
#    from pywhispercpp.model import Model
#    model = Model("base")          # fetches & caches ggml-base.bin (~142 MB)
#    for seg in model.transcribe("audio.wav"):
#        print(seg.text)

# --- Or get ggml-base.bin manually (no pip) ---
# From a cloned whisper.cpp checkout:
bash ./models/download-ggml-model.sh base       # -> models/ggml-base.bin

# Or download the file directly from Hugging Face:
curl -L -o ggml-base.bin \
  https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-base.bin

Head-to-Head Benchmark Table

Reference Benchmark: GPU vs CPU Timings from faster-whisper

Real-Time Transcription Setup

• whisper.cpp stream mode: Run ./stream -m models/ggml-small.bin --step 3000 --length 10000 -t 4. Processes 3-second audio chunks; ~0.5–1.5 second lag with the small model. No Python needed.
• faster-whisper VAD pipeline: Use vad_filter=True in model.transcribe(). Silero VAD automatically segments audio at silence boundaries — more natural chunks than fixed-length windows.
• Practical latency: 0.5–2 seconds behind live speech with small or medium models. Use tiny for the lowest latency (< 0.5 seconds, but higher WER).
• Model selection for real-time: small or base is the practical sweet spot — fast enough to keep up with speech, accurate enough for clean audio. Avoid large-v3 for real-time unless you have a dedicated GPU.
• Microphone input: whisper.cpp reads raw audio via SDL2 or portaudio. faster-whisper reads audio arrays from any Python audio library (sounddevice, pyaudio, soundfile).
• Stability: whisper.cpp stream mode can produce repeated tokens ("hallucinate" short fillers) on silence. Suppress with --suppress-blank and --no-speech-threshold.

Apple Silicon: whisper.cpp Wins

• Core ML export: Run ./models/generate-coreml-model.sh large-v3 to export the encoder to Core ML format. This offloads encoder inference to the Apple Neural Engine.
• M5 Pro benchmark (large-v3, Metal): ~10× real-time. 60 minutes of audio transcribes in ~6 minutes. Note: M5 Pro shipped March 2026 — these are early community benchmarks. Performance may improve with whisper.cpp updates optimizing for the M5 Neural Engine.
• M3 MacBook Air benchmark (large-v3, Metal): ~7× real-time. 60 minutes in ~8.5 minutes.
• Memory: Unified memory means no separate VRAM — a 16 GB M5 Pro can comfortably run large-v3 (~3 GB) alongside other processes.
• faster-whisper on Mac: CPU-only, int8. Large-v3 at ~3× real-time. Usable for batch overnight transcription but not for real-time or time-sensitive workflows.
• Recommendation: Use whisper.cpp for all Mac STT work. Add pywhispercpp if you need Python integration while retaining Metal acceleration.

NVIDIA GPU: faster-whisper Wins

• RTX 4070 (12 GB) benchmark (large-v3 int8): ~12× real-time, ~2.5 GB VRAM.
• RTX 3060 (12 GB) benchmark (large-v3 int8): ~8× real-time, ~2.5 GB VRAM.
• RTX 4060 (8 GB) benchmark (large-v3 int8): ~7× real-time, ~2.5 GB VRAM — easily fits.
• int8 vs float16: int8 is ~2× faster and uses ~40% less VRAM with negligible accuracy loss. Always use compute_type="int8" on NVIDIA.
• Batch processing: faster-whisper's batched=True parameter enables parallel processing of multiple audio files, maximizing GPU utilization for large transcription jobs.
• Python pipeline integration: faster-whisper slots directly into LangChain, Haystack, and custom Python pipelines. No subprocess overhead vs. wrapping whisper.cpp.

Older NVIDIA GPUs (GTX 970): int8 Falls Back to float32

• What still works: CTranslate2's prebuilt binaries support NVIDIA GPUs with compute capability 3.5 or higher, so a GTX 970 (5.2) can run faster-whisper on CUDA, provided your installed CUDA and cuDNN libraries match the release (see Common Issues).
• Where GPU int8 works: int8 needs compute capability 7.0 or higher (Volta and RTX 20-series onward) or exactly 6.1 (GTX 1050 to 1080 Ti, int8_float32). Compute capability 6.0, 6.2 and everything below fall back to float32; float16 needs 7.0 or higher.
• Memory consequence: float32 stores 4 bytes per parameter against 1 byte for int8, so by parameter count large-v3 (1.55B) needs roughly 6 GB for weights alone — more than the GTX 970's 4 GB. Small (244M, about 1 GB in float32) fits comfortably; medium (769M, about 3 GB) is borderline.
• Speed: We have not benchmarked a GTX 970, so there is no measured figure here. Expect float32 throughput well below the RTX 4070 int8 numbers in this guide, and test tiny, base and small on your own audio.
• Check what your GPU supports: ctranslate2.get_supported_compute_types("cuda") lists the compute types CTranslate2 accepts on your card.
• Fallbacks: CPU int8 (device="cpu", compute_type="int8") works on any modern x86 CPU, and whisper.cpp's quantized GGML models (for example ggml-large-v3-q5_0.bin, about 3–4 GB) are the usual way to fit larger models into 4 GB — verify the CUDA or Vulkan build on your card.

import ctranslate2
from faster_whisper import WhisperModel

# Which compute types does this GPU accept?
print(ctranslate2.get_supported_compute_types("cuda"))

# On compute capability < 6.1 (e.g. GTX 970), request float32 explicitly so nothing is converted silently
model = WhisperModel("small", device="cuda", compute_type="float32")

Whisper on Raspberry Pi 5: tiny, base & small

• Whisper small on Pi 5: Roughly 0.4–0.6× real-time on CPU (a 10-minute clip takes ~17–25 minutes). Accurate at 3.4% WER but too slow for live captioning — use it for overnight/batch transcription.
• Whisper base on Pi 5: Around real-time to ~2× with multithreading (-t 4). The practical sweet spot for live transcription on the Pi, with acceptable 5.0% WER.
• Whisper tiny on Pi 5: Fastest option, comfortably faster than real-time for live use, at the cost of higher 7.6% WER. Best for wake-word and short-command recognition.
• Use quantized models: bash ./models/download-ggml-model.sh small-q5_0 trims RAM and speeds inference with minimal accuracy loss — important on the Pi 5's shared memory.
• faster-whisper on Pi 5: Installable (pip install faster-whisper, CPU int8) and competitive for batch jobs, but whisper.cpp's lighter C binary is the more common embedded choice.
• Thermals: Sustained transcription pushes the Pi 5 CPU hard — use active cooling to avoid thermal throttling that erodes these numbers.

When to Use Which

Beyond whisper.cpp and faster-whisper

• WhisperX: Built on top of faster-whisper, WhisperX adds word-level timestamps and speaker diarization — identifying which speaker said which words. Best for meeting transcription with speaker labels, podcast editing, and interview transcripts. Install with pip install whisperx and provide a Hugging Face token for the diarization model.
• insanely-fast-whisper: A Hugging Face Transformers pipeline wrapper that adds Flash Attention 2 support for significantly faster GPU inference than standard faster-whisper on NVIDIA hardware. Best for batch processing large audio archives on NVIDIA GPUs. Requires a Flash Attention 2-compatible GPU (Ampere or newer: RTX 3000+, A100, H100).

Common Issues and Fixes

• CUDA version mismatch: Current faster-whisper releases need cuBLAS for CUDA 12 and cuDNN 9. If you are stuck on older libraries, pin ctranslate2: pip install --force-reinstall ctranslate2==4.4.0 (CUDA 12 with cuDNN 8) or 3.24.0 (CUDA 11 with cuDNN 8). Check your CUDA version with nvcc --version.
• Metal model export fails: Ensure Xcode Command Line Tools are installed — run xcode-select --install. The Core ML export script requires the coremltools Python package: pip install coremltools.
• Hallucination on silence: Both tools can produce repeated filler tokens on silent audio segments. Use --no-speech-threshold 0.6 in whisper.cpp stream mode, or vad_filter=True in faster-whisper's model.transcribe() to skip silent segments automatically.
• Out of memory on large-v3: Switch to int8 quantization in faster-whisper (compute_type="int8") — reduces VRAM from ~5 GB (float16) to ~2.5 GB. In whisper.cpp, use the quantized GGML variant (e.g., ggml-large-v3-q5_0.bin) which cuts memory to ~3–4 GB.
• Garbled output on non-English audio: Do not use .en model variants (tiny.en, base.en) for non-English speech — they are English-only. Use the multilingual models (base, small, medium, large-v3) and set the language explicitly: -l de in whisper.cpp or language="de" in faster-whisper.
• Slow CPU inference: Ensure your CPU supports AVX2 instructions (required for optimized CPU inference). Check with grep avx2 /proc/cpuinfo on Linux or sysctl machdep.cpu.features on Mac. CPUs without AVX2 fall back to generic SIMD and will be 2–3× slower.

Frequently Asked Questions

Sources

• whisper.cpp on GitHub — Source, build instructions, model download scripts, and Metal/Core ML setup guide.
• faster-whisper on GitHub — Source, Python API documentation, and benchmark results.
• distil-whisper/distil-large-v3 on Hugging Face — Model card, benchmark results, and usage instructions for the distilled Whisper variant.
• WhisperX on GitHub — Word-level timestamps and speaker diarization built on faster-whisper.
• insanely-fast-whisper on GitHub — Flash Attention 2 Whisper pipeline for maximum NVIDIA GPU throughput.
• OpenAI Whisper on GitHub — Original Whisper model, paper, and model cards for all sizes.
• OpenAI Whisper paper (Radford et al., 2022) — "Robust Speech Recognition via Large-Scale Weak Supervision." Source of WER figures.
• CTranslate2 documentation — Quantization details, hardware support, and int8 optimization rationale.
• CTranslate2 quantization documentation — Which compute types each GPU compute capability supports (basis of the GTX 970 section).
• faster-whisper README benchmarks — 13-minute GPU and CPU timings used in the reference benchmark, plus current CUDA 12 / cuDNN 9 requirements.
• openai/whisper-large-v3-turbo on Hugging Face — Model card: 809M parameters, 4 decoder layers, MIT license.
• Bill Mill: updated mlx_whisper vs. whisper.cpp benchmark — Independent large-v3-turbo timing comparison on Apple Silicon.
• whisper.cpp releases — Release notes for v1.9.4 and v1.9.5.

Related Reading

• Whisper.cpp Review (2026) — a deep dive on whisper.cpp alone: history, install walkthrough, real CLI commands, license, and when to pick faster-whisper instead.
• faster-whisper Review — the CTranslate2-based counterpart, reviewed in the same depth.
• Local TTS and Voice Cloning 2026: Piper vs Coqui vs XTTS v2 — The TTS companion: turn transcribed text back into speech, offline.
• Build a Fully Offline Voice Assistant in 2026 — Connect Whisper STT to a local LLM and Piper TTS into a complete voice pipeline.
• Local Multimodal AI Pipeline 2026 — Add vision models alongside STT and TTS for a full multimodal local stack.
• Local LLM Hardware Guide — GPU and RAM requirements for the full voice + LLM pipeline.
• Build a Local Voice Assistant on Your Phone — Mobile version: WhisperKit on iPhone, Layla on Android.