What Is MLX-Audio?
MLX-Audio turns Apple's MLX framework into a speech toolbox. One package provides commands and Python functions that fetch a speech model from Hugging Face, run it on the Mac's GPU through Metal, and write audio or text back. Its README describes it as "the best audio processing library built on Apple's MLX framework", which is the project's own positioning, not an independent ranking.
- Product type: a Python library with command-line entry points such as
mlx_audio.tts.generateandmlx_audio.server, not a downloadable desktop app - Maintainer: the GitHub user Blaizzy with community contributors; no company, pricing page or commercial edition was found
- Repository and documentation: github.com/Blaizzy/mlx-audio and blaizzy.github.io/mlx-audio
- Related project: mlx-audio-swift brings on-device text-to-speech to macOS and iOS apps
- Used elsewhere: the vllm-mlx review notes that vllm-mlx builds on mlx-audio for its audio features
What Can You Do With MLX-Audio?
The README groups its supported models into text-to-speech, speech-to-text, voice activity detection, speech-to-speech and music generation. Here is what each group does, per the project's own model tables.
- Text-to-speech — about two dozen model families, including Kokoro, Qwen3-TTS, Chatterbox, VoxCPM2, Higgs Audio, Voxtral TTS and OmniVoice
- Voice cloning and voice design — several models accept a reference recording (for example
--ref_audiowith CSM), and some build a voice from a text description - Speech-to-text — Whisper, Parakeet, Qwen3-ASR and NVIDIA's streaming Nemotron models; VibeVoice-ASR adds speaker labels and timestamps for recordings of up to 60 minutes
- Voice activity and speaker detection — Silero VAD plus Sortformer and Nemotron diarization models
- Speech-to-speech — source separation (SAM-Audio), noise removal (DeepFilterNet, MossFormer2) and full-duplex voice chat (NemotronLabs VoiceChat)
- Music generation — MiniMax Music 3 for songs with lyrics at 44.1 kHz stereo
- Streaming and speed control —
--streamplays audio while it is generated, and models that support it take a speed parameter - Quantisation —
python -m mlx_audio.convertshrinks models to 3-, 4-, 6- or 8-bit, or MXFP4, for smaller downloads and lower memory use
Usage Examples: Three Ways to Use MLX-Audio
These workflows use commands taken from the project's README.
Generate speech from the command line
Run mlx_audio.tts.generate --model mlx-community/Qwen3-TTS-12Hz-0.6B-CustomVoice-8bit --text 'Hello, world!' --voice Vivian --play to synthesise and play a sentence. Add --output_path ./my_audio to save files, or --join_audio to merge several segments into one file.
Transcribe a recording from Python
Call generate_transcription(model="mlx-community/whisper-large-v3-turbo-asr-fp16", audio="audio.wav") from mlx_audio.stt.generate and read result.text. Swapping the model name switches to Parakeet, Qwen3-ASR or another listed engine.
Run a local OpenAI-style speech server
Start mlx_audio.server --host 127.0.0.1 --port 8000, then post to /v1/audio/speech for synthesis or /v1/audio/transcriptions for transcription. The README's own example binds 0.0.0.0, which exposes the server to your whole network; 127.0.0.1 keeps it on the Mac.
Install MLX-Audio
MLX-Audio installs from the command line; there is no .dmg or other installer. The easiest path is one pip command, so a terminal and Python 3.10 or newer are required.
Source | Link |
|---|---|
| Website (documentation) | blaizzy.github.io/mlx-audio |
| Install (pip) | pip install mlx-audio |
| Install (uv, command-line tools only) | uv tool install --force mlx-audio --prerelease=allow |
| GitHub repository (source code, MIT) | github.com/Blaizzy/mlx-audio |
| PyPI package | pypi.org/project/mlx-audio |
| Swift package (macOS and iOS) | github.com/Blaizzy/mlx-audio-swift |
| Privacy policy | None found on the website or in the repository; the library itself runs locally |
An Apple Silicon Mac is required. For the web UI or development, the README clones the repository and runs `pip install -e ".[dev, server]"`, then starts the UI with npm.
MLX-Audio Licence and Pricing
MLX-Audio is free. There is no account, licence key, paid tier or usage cap, and neither the README nor the documentation mentions a commercial edition.
- The library is MIT-licensed, which permits commercial use, modification and redistribution with the licence notice kept
- Speech models are downloaded separately and carry their own licences, some more restrictive than MIT, so read each model card before shipping a product
- The real costs are an Apple Silicon Mac and disk space for the models you download
- No hosted or managed version was found; MLX-Audio runs only on hardware you control
Who Should Use MLX-Audio?
MLX-Audio suits people who are comfortable in a terminal and want speech models on a Mac without a cloud account.
Use MLX-Audio if you
- Own an Apple Silicon Mac and want text-to-speech, transcription and voice tools in one Python package
- Want to try new open speech models quickly, since the README already lists dozens of model families
- Have code written for OpenAI's speech endpoints and want to point it at a local server
- Build Mac or iOS apps and want the companion Swift package for on-device speech
Look elsewhere if you
- Use Windows, Linux or an Intel Mac, where MLX-Audio does not run; try faster-whisper or Piper
- Want a point-and-click transcription app rather than a library; see MacWhisper
- Need a vendor-backed product with a support contract, which this independent project does not offer
Competitors and Alternatives
MLX-Audio sits among local speech tools that differ mainly in platform, model choice and whether they are libraries or apps. The four below have their own PromptQuorum reviews.
Tool | Best known for | Link |
|---|---|---|
| whisper.cpp | Local Whisper speech recognition in C/C++ for CPU and GPU | whisper.cpp review |
| faster-whisper | Fast Whisper transcription through CTranslate2 | faster-whisper review |
| Piper | Lightweight multilingual text-to-speech that runs on CPU | Piper review |
| Coqui TTS | Open-source voice synthesis with several model architectures | Coqui TTS review |
This list reflects tools commonly compared in local speech work, not a PromptQuorum ranking. Verify each tool's current features before choosing.
Frequently Asked Questions
Which languages does MLX-Audio support?
It depends on the model. The README lists Kokoro with eight languages, Chatterbox with 23, VoxCPM2 with 30, OmniVoice with 646+, and Whisper with 99+ for transcription.
Do I need ffmpeg?
Only for MP3, FLAC, OGG, Opus or Vorbis output. WAV works without it, and brew install ffmpeg adds the rest.
Can MLX-Audio clone a voice?
Yes, with models that accept a reference clip. Cloning a real person's voice without their consent can be unlawful in many places, so get permission and check the model's licence.
Does the server need an OpenAI API key?
The README's example requests send no key. The endpoints copy OpenAI's request shape, but nothing is sent to OpenAI.
Can I use it inside an iPhone app?
Not the Python package. The separate mlx-audio-swift package provides on-device text-to-speech for macOS and iOS.