Key Takeaways
- Free, open-source, Apache-2.0 license, copyright Exo Technologies Ltd, per the repository's own LICENSE file β verified 2026-09-12
- Automatic device discovery β devices running exo on the same network find each other with no manual configuration, per exo's own README
- Topology-aware auto parallel: exo splits the model across devices based on a real-time view of each device's resources and the network latency/bandwidth between links
- Tensor parallelism on top of that sharding β up to a 1.8x speedup on 2 devices and 3.2x on 4 devices, per exo's own README
- RDMA over Thunderbolt 5 support (macOS 26.2+), which exo describes as a 99% reduction in inter-device latency, per its own documentation
- Uses MLX, Apple's array-computing framework, as the inference backend and MLX distributed for cross-device communication
- Compatible with OpenAI Chat Completions, OpenAI Responses, Claude Messages, and Ollama APIs, so existing clients can point at an exo cluster largely unmodified
- 47,375 GitHub stars, 3,508 forks, verified via the GitHub API on 2026-09-12
- macOS (Apple Silicon) is the primary, tested platform; Linux runs CPU-only today (GPU support is listed as "under development" in exo's own PLATFORMS.md); Windows is not currently supported
π In One Sentence
exo is a free, open-source (Apache-2.0) framework that pools multiple Apple Silicon Macs β and, with CPU-only support, Linux machines β into a single distributed inference cluster, using automatic device discovery and topology-aware model sharding to run models too large for any single device.
π¬ In Plain Terms
Instead of buying one very expensive machine with enough memory to load a huge model, or renting cloud GPU time, exo lets you connect several Macs (or Linux boxes) you already own over the network so they act as one larger computer, automatically splitting the model's layers across all of them.
πNote: This review is the companion piece to exo's entry in the Local LLM Software Directory β see that page for how exo compares at a glance to dozens of other local AI tools.
What Is exo?
exo is a framework for connecting multiple devices you already own into one AI inference cluster, so that models too large to fit in any single device's memory can still run, by splitting the model across the group. It is maintained by exo labs and its own repository description reads simply: "Run frontier AI locally."
- Core function: automatic device discovery plus topology-aware model sharding β point several devices at the same cluster and exo works out how to split the model across them
- Inference backend: MLX, Apple's open-source array-computing framework, used both for running the model and for the MLX-distributed communication layer between devices
- Deployment model: run from source (macOS or Linux), via the Nix package manager, or β macOS only β as a background app you install with Homebrew or a downloadable
.dmg - Developer: exo-explore / exo labs on GitHub; the repository's LICENSE file lists the copyright holder as Exo Technologies Ltd
- Canonical repository: github.com/exo-explore/exo, created June 2024 per GitHub's own repository metadata, with a push recorded as recently as 2026-08-25 at the time of this review
Install exo: Setup Options
exo's own README documents three ways to run it: building from source on macOS or Linux, running it through the Nix package manager, or β macOS only β installing a prebuilt background app. There is no pip install exo package; source installs use uv for Python dependencies.
- This starts the exo dashboard and OpenAI-compatible API at
http://localhost:52415, per exo's own README - On Linux, the equivalent commands use
uv sync --extra mlx-cpu(or--extra mlx-cuda13/--extra mlx-cuda12where applicable); exo's own README states that on Linux, exo currently runs on CPU only, with GPU support "under development" - If Nix is already installed,
nix run .#exoskips most of the manual dependency setup on macOS - On macOS, exo also ships a prebuilt background app requiring macOS Tahoe 26.2 or later β install it with
brew install --cask exoor download EXO-latest.dmg directly, per exo's own documentation - Two configuration flags exist for source installs, per the README:
--no-worker(run a coordinator-only node without local inference β useful for a machine with good network connectivity but limited GPU/memory) and--legacy-daemon(run as a background daemon for older init systems)
git clone https://github.com/exo-explore/exo
cd exo/dashboard && npm install && npm run build && cd ..
uv sync --extra mlx
uv run exoHow exo's Clustering Actually Works
exo's own README documents four mechanisms that together let a group of separate devices behave as one inference cluster, rather than requiring you to manually decide which layers run where.
- exo's own caveats for RDMA: every device in the cluster must be directly cabled to every other device with Thunderbolt-5-rated cables, and all devices must run the exact same macOS version, including beta build numbers
- A
--no-workerflag lets a device join a cluster as a coordinator only, without running inference locally β useful for a machine with strong networking but weak compute - The
EXO_OFFLINEenvironment variable runs a cluster without an internet connection, using only models already cached locally
Supported Models and Platforms
exo loads models from the Hugging Face Hub through its MLX inference backend, and exo's own README does not document support for GGUF-format models β that distinguishes it from llama.cpp-based tools like Ollama or LM Studio, which are GGUF-first.
Performance: What the Benchmarks Show
exo's own README links to third-party benchmarks rather than publishing first-party numbers for every configuration; PromptQuorum has not independently re-run these tests and reports them here as exo's own cited sources, not as PromptQuorum's own measurements.
- exo's README shows a dashboard screenshot of 4 Γ 512 GB M3 Ultra Mac Studios running DeepSeek V3.1 (8-bit) and Kimi-K2-Thinking (4-bit) simultaneously, as an illustration of the memory pooling exo enables
- A cited benchmark from Jeff Geerling's blog covers Qwen3-235B (8-bit) run across 4 Γ M3 Ultra Mac Studios with tensor-parallel RDMA
- The same cited source also covers DeepSeek V3.1 671B (8-bit) and Kimi K2 Thinking (native 4-bit) on the same 4-device cluster configuration
- exo's own README states the general multipliers of up to 1.8x (2 devices) and 3.2x (4 devices) from tensor parallelism specifically, separate from the third-party large-model benchmarks above
Usage Examples
These are workflows built from exo's own documented features above, not hypothetical use cases.
Who Should Use exo?
Whether exo fits depends heavily on the hardware you already have and whether it is macOS-based β exo's clustering is built around and tested on Apple Silicon first.
exo vs. Other Distributed Inference Tools
exo sits in the distributed/cluster local-inference segment, alongside other projects that split a model across multiple machines rather than serving it from one. Here is how it compares β see the Local LLM Software Directory for the full catalog.
- GPUStack β an open-source GPU cluster manager (Apache-2.0) that pools GPUs across Linux, Windows, and macOS and can serve models through Ollama, vLLM, or llama.cpp as backends; a broader, backend-agnostic cluster manager compared to exo's MLX-specific, Apple-Silicon-first design. GPUStack has its own entry in PromptQuorum's Local LLM Software Directory.
- llama.cpp's RPC backend β llama.cpp ships an experimental
ggml-rpc-serverthat offloads computation to remote hosts for distributed inference; its own maintainers label it a "proof-of-concept" that is "fragile and insecure" and warn never to expose it on an open network, a materially different maturity and security posture than exo's documented clustering. - Petals β a research project from the BigScience workshop that runs large models "BitTorrent-style" across a public, volunteer-run swarm of consumer GPUs, rather than a private cluster of devices you own; a different trust model from exo's LAN-based clustering, aimed at collaborative inference across strangers' hardware rather than your own local network.
Common Mistakes When Evaluating exo
Most confusion about exo comes from assuming it behaves like a single-device chat app, or expecting Linux/Windows to match macOS's feature set.
Frequently Asked Questions
What is exo?
exo (exolabs.net, source at github.com/exo-explore/exo) is an open-source, Apache-2.0 licensed framework for connecting multiple devices β primarily Apple Silicon Macs β into one distributed AI inference cluster, so models too large for any single device can still run by being split across the group.
Is exo free?
Yes. exo is open source under the Apache-2.0 license, verified against the repository's own LICENSE file β there is no separate paid tier or field-of-use restriction beyond the standard Apache-2.0 terms.
What license does exo use? Has it changed?
exo is licensed under the Apache License 2.0, copyright Exo Technologies Ltd, per the current LICENSE file in the repository, checked directly on 2026-09-12. PromptQuorum found no additional field-of-use or commercial-use restriction layered on top of the standard Apache-2.0 text in the current repository.
How does exo's clustering actually work?
Per exo's own README, devices running exo automatically discover each other on the same network, exo builds a real-time view of each device's memory and the network links between them, and it splits a model's layers across the cluster accordingly, adding tensor parallelism for a further speedup. On supported macOS hardware, it can additionally use RDMA over Thunderbolt 5 for lower-latency links between devices.
What model formats does exo support?
exo's own README documents loading models from the Hugging Face Hub through its MLX inference backend; it does not document support for GGUF-format models, which distinguishes it from GGUF-first tools like llama.cpp or Ollama.
Does exo run on Windows?
Not currently. exo's own PLATFORMS.md lists Windows CUDA and Windows CPU support only under a "Longer term" roadmap heading, separate from its near-term plans.
Does exo run on Linux?
Yes, but CPU-only today. exo's own README and PLATFORMS.md are explicit that GPU support for Linux (CUDA and Vulkan) is under development and not yet shipped, as of this review.
How do I install exo?
Build from source with git clone https://github.com/exo-explore/exo, build the dashboard, run uv sync --extra mlx (macOS) or uv sync --extra mlx-cpu (Linux), then uv run exo. On macOS, you can alternatively install a prebuilt background app with brew install --cask exo or by downloading EXO-latest.dmg β it requires macOS Tahoe 26.2 or later.
How many GitHub stars does exo have?
exo had 47,375 stars and 3,508 forks on GitHub, verified via the GitHub API on 2026-09-12. Check the live repository for the current count, since star counts change continuously.
Who develops exo?
exo is developed by exo-explore / exo labs on GitHub; the repository's LICENSE file lists the copyright holder as Exo Technologies Ltd. The repository was created in June 2024, per GitHub's own metadata.