What Is oMLX?
oMLX is a local inference server for Apple Silicon Macs that turns Apple's MLX machine-learning framework into a full serving stack — an OpenAI/Anthropic-compatible API, a multi-model manager, a paged SSD cache, and a native macOS menu-bar app to control it all, without requiring a terminal for day-to-day use. Its own GitHub description positions it as "an LLM inference server optimized for Apple Silicon Macs, featuring continuous batching, tiered KV caching, and a native macOS menu bar management interface."
- Core function: a background inference server that loads open-weight models and serves them over a local HTTP API, managed either from a menu-bar app or the
omlxCLI - Inference engine: built on Apple's MLX framework and the mlx-lm library's
BatchGenerator, per oMLX's own documentation, with an original continuous-batching and paged-cache layer built on top - Foundation: per the developer's own announcement, oMLX was originally built on top of vllm-mlx as a starting point, with original implementations added for SSD cache tiering, continuous batching, vision-language model support, the Anthropic-compatible API, and the native macOS interface
- Developer: jundot (contact listed as junkim.dot@gmail.com in the repository), an independent maintainer — this review found no evidence of a company or funding round behind oMLX
- Canonical repository: github.com/jundot/omlx — several identically-named forks exist under other GitHub usernames (for example mkmsyk/omlx, dannysl/omlx); those are not the original project
oMLX's Origin and Growth
oMLX was first announced publicly by its developer, jundot, in a March 2026 post on Apple MLX's own GitHub discussions board. The announcement framed oMLX as a fix for a specific, concrete problem: coding agents that reuse a long, growing context on Apple Silicon could see multi-tens-of-seconds delays reloading that context on every request, because MLX-based servers of the time typically kept key-value cache state only in RAM and lost it whenever a server restarted or a context aged out. oMLX's paged SSD cache persists that state to disk instead, so a previously processed prefix can be restored from SSD rather than recomputed from the model. According to the developer's own announcement, at launch oMLX had already accumulated roughly 110 GitHub stars, built on top of vllm-mlx as a starting point with newly written code for SSD tiering, continuous batching, vision-language model support, an Anthropic-compatible API, and the native macOS interface. As of this review's publish date, oMLX's GitHub repository shows more than 23,000 stars — fast growth for a project that was under eight months old at the time of writing.
- Announcement: March 4, 2026, on github.com/ml-explore/mlx/discussions/3203, posted by the project's own developer, jundot
- Starting point: vllm-mlx, with the developer's own SSD-tiering, continuous-batching, VLM, Anthropic-API, and native-UI code added on top
- Growth: from roughly 110 GitHub stars at the March 2026 announcement to more than 23,000 stars as of this review, per oMLX's own GitHub repository
- License history: the repository's LICENSE file carries a 2025 copyright notice for "oMLX contributors," ahead of the project's public March 2026 announcement — consistent with private development before the public launch
What Can You Do With oMLX?
oMLX's feature set centers on turning MLX model inference into a persistent, multi-model local server, rather than a one-off script. Here is what each part actually does, per oMLX's own GitHub README and omlx.ai documentation.
- Tiered key-value cache — active context blocks stay in a hot RAM tier; inactive blocks move to a cold SSD tier in safetensors format, with prefix sharing and copy-on-write, and the cache survives server restarts instead of being discarded
- Continuous batching — concurrent requests are handled through mlx-lm's
BatchGenerator, with a configurable maximum concurrency, rather than processing one request at a time - Multi-model serving — load LLMs, vision-language models (VLMs), embedding models, and rerankers side by side, with LRU eviction, manual load/unload, model pinning, and a configurable per-model time-to-live
- OpenAI- and Anthropic-compatible API —
/v1/chat/completions,/v1/completions,/v1/embeddings, and/v1/rerankfor OpenAI-format clients, plus/v1/messagesfor Anthropic-format clients, all served fromhttp://localhost:8000 - Vision-language and OCR support — multi-image chat with base64, URL, or file inputs, tool calling with vision context, and auto-detection of dedicated OCR models
- Tool calling and MCP — JSON-schema tool calling with automatic chat-template detection for Llama, Qwen, DeepSeek, Gemma, GLM, MiniMax, Mistral, and other model families, plus Model Context Protocol (MCP) configuration
- Native macOS menu-bar app — a Swift/SwiftUI app (explicitly not Electron, per the project's own documentation) for starting, stopping, and monitoring the server without a terminal, with local usage analytics (per-model totals, an hourly usage heatmap) and automatic restart on crash
- Admin dashboard — a web UI at
/admin, reachable from a browser once the server is running, for model management, a built-in chat interface, one-click benchmarking, a model downloader that filters Hugging Face by MLX or GGUF format, and one-click setup for coding-agent integrations including OpenClaw, OpenCode, Codex, Hermes Agent, Copilot, Pi, Mistral Vibe, and DeepSeek Harness - External engine for GGUF and NVFP4 models — a per-model setting that serves selected Qwen checkpoints, including GGUF files, through the separate Splash Metal engine in its own process; the README lists the supported models and requires Splash 1.4.0 or later
- Experimental distributed inference — splits one downloaded language model across Macs with unequal memory using MLX pipeline ranks over Ring or Thunderbolt RDMA/JACCL; source builds only and marked experimental in oMLX's own README (see the distributed section below)
oMLX Distributed Inference and Clustering
oMLX's distributed inference is an experimental, source-build-only feature that splits one downloaded language model across multiple Macs, including Macs with unequal memory. Per the oMLX README, it uses MLX pipeline ranks over Ring or Thunderbolt RDMA/JACCL networking, and it is not part of the standard macOS app or Homebrew install.
- Status: labelled "Experimental Multi-Mac Inference" in the README; the project documents setup, security boundaries, current limitations, and a physical-hardware validation checklist in a separate doc, distributed-cluster.md
- Availability: source builds only — the macOS .dmg app and Homebrew install paths in the table above do not include it
- Cluster dashboard: read-only peer discovery with strict SSH and runtime verification, per the README
- Profiles: interactive, balanced, and throughput profiles, with coalesced batching and prompt-cache affinity, per the README
- Networking: Ring, or Thunderbolt RDMA/JACCL between Macs
Usage Examples: Three Ways to Use oMLX
These are concrete workflows built from oMLX's documented features above — not hypothetical use cases.
Run the menu-bar app and point Claude Code at it
Install the macOS app, launch it from the menu bar, and start the server. oMLX serves an Anthropic-compatible /v1/messages endpoint on http://localhost:8000, so once you point Claude Code at that address it can use whichever local model you have loaded in oMLX instead of a cloud API. The README documents Claude Code-specific handling: oMLX reports the model's real context window so auto-compact works, and SSE keep-alive prevents read timeouts during long prefills. The dashboard at http://localhost:8000/admin offers one-click setup for other agents such as OpenCode or Codex.
Start the CLI server directly and reuse an OpenAI-client integration
From a terminal, run omlx serve --model-dir ~/models (or omlx start to run it in the background). Once the server is listening on http://localhost:8000, point any existing OpenAI-client code or tool at that address instead of api.openai.com, using /v1/chat/completions — the same pattern used by other local OpenAI-compatible servers — to reuse code you already have rather than rewriting it for a new API shape.
Keep a long coding-agent context fast across a work session
Run omlx serve with --paged-ssd-cache-dir pointed at a fast local drive and a --hot-cache-max-size sized for your Mac's RAM. As you work through a long agentic coding session that repeatedly revisits the same large context, previously processed blocks that would otherwise be evicted from RAM are instead recovered from the SSD tier, which oMLX's own documentation credits with reducing multi-tens-of-seconds reload waits down to a few seconds on long contexts.
Install oMLX: macOS App, Homebrew, and CLI
oMLX installs as a macOS app, via Homebrew, or from source — there is no Windows or Linux build, because it depends on Apple's MLX framework, which only targets Apple Silicon. All three methods install the same underlying omlx command-line tool. Links below are from the official GitHub README — always verify against that page directly, since install instructions can change between releases.
macOS app (.dmg)
- Command / Link:
- Download from GitHub Releases — one .dmg for macOS 15 (Sequoia) and one for macOS 26/27 — and drag to Applications. Includes auto-update and installs a CLI shim at
~/.omlx/bin/omlx.
Homebrew
- Command / Link:
brew tap jundot/omlx https://github.com/jundot/omlx, thenbrew install jundot/omlx/omlx(add--with-custom-kernelto build the native kernels; needs full Xcode), thenomlx start
From source
- Command / Link:
git clone https://github.com/jundot/omlx.git, thencd omlx, thenmake install
Run the server directly
- Command / Link:
omlx serve --model-dir ~/models(foreground) oromlx start/omlx stop/omlx restart(background)
System requirements per the official README: macOS 15.0 (Sequoia) or later, Apple Silicon (M1 through M5), and Python 3.11–3.13. There is no Intel Mac, Windows, or Linux build. GLM-5.2, MiniMax M3, and Qwen3.5 need native custom kernels and fall back to much slower generic paths without them, per the README; the .dmg ships them precompiled, while make install and Homebrew's --with-custom-kernel build them and need full Xcode with the Metal toolchain. Server configuration flags — --memory-guard {safe|balanced|aggressive}, --memory-guard-gb, --paged-ssd-cache-dir, --hot-cache-max-size, --max-concurrent-requests, --mcp-config, --hf-endpoint, --api-key, and --host — persist to ~/.omlx/settings.json once set, and can be overridden per-run with CLI flags.
oMLX Pricing: Is oMLX Really Free?
Yes — oMLX has no paid tier. Neither the GitHub repository nor omlx.ai has a pricing page, and the LICENSE file is a plain, unmodified Apache License 2.0 with a 2025 copyright notice for "oMLX contributors" — it applies to the entire application, with no clause gating any feature behind payment.
- No subscription, no paid tier, no usage limits imposed by oMLX itself
- No account or sign-up required to install or run the app
- The application, the menu-bar app, the CLI, and the admin dashboard are all covered by the same Apache License 2.0 terms
- Because oMLX only runs local models through MLX, there is no per-token or per-request cost from a cloud provider either — the only cost is the electricity and hardware you already have
oMLX vs. mlx-lm and MLX
MLX is Apple's machine-learning framework; oMLX is a server product built on it, not a competing framework — so "oMLX vs MLX" is a question of layers, not alternatives. mlx-lm is the Python library and simple CLI that Apple's MLX ecosystem provides for running and serving models — and oMLX is built directly on top of it, using its BatchGenerator for continuous batching. The two are not really competitors so much as different layers: mlx-lm is the foundation, oMLX is a full serving product built on that foundation.
Aspect | oMLX | |
|---|---|---|
| Layer | Server product on top of MLX and mlx-lm | Python library on top of the MLX framework |
| What it is | A packaged inference server: menu-bar app + CLI + admin dashboard | A Python library and lightweight CLI for running/serving MLX models |
| KV caching | Tiered RAM + SSD paged cache, persists across restarts | In-memory cache only, per its own documented server; no SSD tiering |
| Multi-model management | LRU eviction, pinning, per-model TTL, load/unload from a UI | Not a built-in feature — scripted or managed manually |
| API compatibility | OpenAI-compatible and Anthropic-compatible endpoints | OpenAI-compatible server mode (mlx_lm.server) |
| Interface | Native macOS menu-bar app plus web admin dashboard | Command-line only, no GUI |
| Coding-agent integrations | One-click setup for OpenClaw, OpenCode, Codex, and others | Not a documented feature; requires manual endpoint configuration |
If you want the simplest possible way to run one MLX model from a script or a quick server command, mlx-lm alone may be all you need. If you want a persistent, multi-model server with a menu-bar UI, SSD-backed caching for long agent contexts, and one-click coding-agent integrations, oMLX is built specifically to add that layer on top of mlx-lm rather than replace it.
oMLX vs. mlx-serve
oMLX and mlx-serve are both free, open-source local inference servers for Apple Silicon with OpenAI- and Anthropic-compatible APIs; the main differences are the runtime (Python/MLX versus a single Zig binary) and model formats (MLX versus MLX plus GGUF). They are separate projects by different developers, so neither is built on the other.
Aspect | oMLX | |
|---|---|---|
| Language / runtime | Python on MLX/mlx-lm, Swift menu-bar app | Zig single binary (~7 MB), no Python runtime |
| Model formats | MLX models (LLM, VLM, embedding, reranker) | MLX plus embedded llama.cpp for GGUF |
| APIs | OpenAI-compatible and Anthropic-compatible | OpenAI, Anthropic, and Ollama-compatible |
| Companion app | Menu-bar app + web admin dashboard | MLX Core menu-bar app (chat, agents, media) |
| License | Apache License 2.0 | MIT (bundled Apache-2.0 components) |
| macOS requirement | macOS 15.0 or later | macOS 26.2 or later |
Pick oMLX if long, repeatedly-revisited coding-agent contexts and its tiered SSD KV cache (which persists across restarts) are the priority; pick mlx-serve if you want a small Python-free binary or GGUF support alongside MLX. Details for mlx-serve are from the mlx-serve review; this review did not benchmark the two against each other, so verify current features in each project's repository.
Who Should Use oMLX?
Whether oMLX fits depends almost entirely on one thing first: whether you are on an Apple Silicon Mac at all, since that requirement has no workaround.
Use oMLX if
- You are running an Apple Silicon Mac (M1 through M5) on macOS 15.0 or later — this is a hard requirement, not a recommendation
- You want a persistent local inference server with a menu-bar app, rather than a script you have to restart manually
- You use a coding agent (Claude Code, OpenClaw, OpenCode, or similar) with long, repeatedly-revisited contexts, and want to reduce reload delays for that specific workflow
- You want both OpenAI-compatible and Anthropic-compatible endpoints from a single local server, rather than running two separate tools
- You want to serve multiple models — LLMs, VLMs, embedding models, rerankers — from one process with automatic memory management
Wait, or look elsewhere, if
- You are on a Windows or Linux machine, or an Intel Mac — oMLX has no build for any of these, because it depends on Apple's MLX framework, which is Apple Silicon-only; see the Local LLM Software Directory for cross-platform alternatives
- You want an officially supported, funded product with a company and an SLA behind it — this review found no evidence of a company or funding behind oMLX; it is maintained by a single named developer, per the project's own GitHub repository
- You want the absolute simplest, most minimal way to run one MLX model — mlx-lm alone, without oMLX's server layer, may be a better fit
- You need a battle-tested, long-track-record project for a production deployment — oMLX was first announced in March 2026, so its public history is still short compared to more established inference servers
oMLX vs. Other Local Inference Tools
oMLX sits in the Apple Silicon / MLX corner of the local inference landscape. Here is how it compares to other tools in that same segment — see the Local LLM Software Directory for the full catalog, and the dedicated oMLX vs. mlx-lm comparison above for the closest head-to-head.
- mlx-lm — the underlying Python library and CLI that oMLX itself is built on top of; the right choice if you want the simplest possible way to run one MLX model without a full server layer. See the dedicated comparison section above.
- mlx-serve — a separate free, open-source Zig inference server for Apple Silicon with OpenAI-, Anthropic-, and Ollama-compatible APIs and GGUF support through llama.cpp; see the oMLX vs. mlx-serve comparison above.
- exo — an open-source tool for clustering multiple Apple Silicon Macs (and other devices) together to run larger models than any single machine could hold; relevant if oMLX's experimental, source-build-only distributed mode isn't enough for your setup. See the exo review.
- LM Studio — a cross-platform (macOS, Windows, Linux) desktop app that also uses MLX as one of its inference engines on Apple Silicon, alongside llama.cpp; a better fit if you need the same tool to work on non-Mac hardware too. See the LM Studio review.
- llamafile — a single-executable local inference approach built on llama.cpp rather than MLX, that runs the same file across macOS, Windows, and Linux with no install step; a useful contrast if cross-platform portability matters more to you than Apple Silicon-specific performance. See the llamafile explained article.
Common Mistakes When Evaluating oMLX
Most confusion about oMLX comes from assuming it works like a cross-platform tool, confusing it with a same-named fork, or expecting Intel Mac support.
Mistake 1: Assuming oMLX runs on Windows or Linux
oMLX is macOS-only, and specifically Apple Silicon-only, because it is built on Apple's MLX framework, which does not target Windows or Linux at all. There is no roadmap item or beta build for either platform documented in the project's repository at the time of this review — if you need a cross-platform tool, see LM Studio or llamafile instead.
Mistake 2: Confusing github.com/jundot/omlx with a same-named fork
Several GitHub repositories with the identical name "omlx" exist under other usernames (for example mkmsyk/omlx, dannysl/omlx). The original, actively developed project — confirmed by the developer's own announcement on Apple MLX's GitHub discussions board — is github.com/jundot/omlx. Always verify you are looking at that specific repository before installing or citing star counts.
Mistake 3: Assuming Intel-based Macs are supported
oMLX's system requirements explicitly list Apple Silicon (M1 through M5); this review found no mention of Intel Mac support anywhere in the official documentation. MLX itself, the framework oMLX is built on, does not target Intel Macs, so this is not something a future oMLX update is likely to add.
Mistake 4: Expecting oMLX to be a chat client rather than a server
oMLX's admin dashboard does include a built-in chat interface for testing, but its primary purpose is serving models over an API for other applications (like Claude Code, OpenCode, or your own scripts) to use — it is closer in category to a local inference server than to a full-featured standalone chat app like Jan or GPT4All.
Frequently Asked Questions
What is oMLX?
oMLX (github.com/jundot/omlx) is a free, open-source local inference server for Apple Silicon Macs. It runs models through Apple's MLX framework and ships as a native macOS menu-bar app plus a omlx command-line tool and background server, with OpenAI- and Anthropic-compatible API endpoints.
Is oMLX free?
Yes. Neither the GitHub repository nor omlx.ai has a pricing page, and the LICENSE file is a plain, unmodified Apache License 2.0 with no paid tier of any kind.
Does oMLX run on Windows or Linux?
No. oMLX is macOS-only, and specifically requires Apple Silicon (M1 through M5) and macOS 15.0 (Sequoia) or later, because it depends on Apple's MLX framework, which does not target Windows, Linux, or Intel Macs.
Does oMLX run on an Intel Mac?
No. oMLX's documented system requirements specify Apple Silicon only. MLX, the framework it is built on, does not support Intel-based Macs.
How is oMLX different from mlx-lm?
mlx-lm is the underlying Python library and lightweight CLI that Apple's MLX ecosystem provides for running models; oMLX is built on top of mlx-lm's BatchGenerator and adds a persistent server, a menu-bar app, a tiered SSD-backed cache, multi-model management, and OpenAI/Anthropic-compatible endpoints. See the full comparison above.
What is the paged SSD cache in oMLX for?
It offloads inactive key-value cache blocks from RAM to your Mac's SSD, in safetensors format, instead of discarding them. When a long-running coding agent revisits a large context, oMLX can restore the previously processed blocks from SSD rather than recomputing them from the model, which its own documentation credits with cutting reload waits on long contexts from tens of seconds down to a few seconds.
Can I use oMLX with Claude Code or other coding agents?
Yes. oMLX's API is compatible with both the OpenAI and Anthropic request formats, its README documents Claude Code-specific handling (real context-window reporting and SSE keep-alive), and its admin dashboard offers one-click setup for OpenClaw, OpenCode, Codex, and others.
Is oMLX open source? What license does it use?
Yes. oMLX's LICENSE file is a plain, unmodified Apache License 2.0, with a 2025 copyright notice for "oMLX contributors."
How do I install oMLX with Homebrew?
Run brew tap jundot/omlx https://github.com/jundot/omlx, then brew install jundot/omlx/omlx, then omlx start. The oMLX README also lists a macOS .dmg app and a from-source install; verify the commands there, since install steps can change between releases.
Does oMLX support distributed inference across multiple Macs?
Experimentally. The oMLX README describes source builds that can split one downloaded model across Macs with unequal memory using MLX pipeline ranks over Ring or Thunderbolt RDMA/JACCL. It is not in the standard app or Homebrew install, and the project documents its limitations in a separate distributed-cluster doc.
What is the latest oMLX version?
v0.7.2, published October 21, 2026. Stable releases are preceded by development and release-candidate builds on the same Releases page, so check github.com/jundot/omlx/releases for anything newer before citing a version.
What is the difference between oMLX and MLX?
MLX is Apple's open-source machine-learning framework for Apple Silicon. oMLX is a local inference server built on MLX and the mlx-lm library, adding a menu-bar app, multi-model management, a paged SSD cache, and OpenAI/Anthropic-compatible APIs.
oMLX vs mlx-serve: which should I use?
Both are free, open-source Apple Silicon inference servers. oMLX is Python-based with a tiered SSD KV cache aimed at long coding-agent contexts; mlx-serve is a single Zig binary that also runs GGUF models through llama.cpp. See the side-by-side table above.
Who develops oMLX?
oMLX is developed by jundot, an independent maintainer (contact listed as junkim.dot@gmail.com in the repository). This review found no evidence of a company or funding round behind the project.
Is github.com/jundot/omlx the real oMLX repository?
Yes. Several identically-named "omlx" repositories exist under other GitHub usernames, but github.com/jundot/omlx is the original project, confirmed by the developer's own announcement on Apple MLX's GitHub discussions board.