Key Takeaways
- oMLX is free and open source; the official GitHub LICENSE is a plain, unmodified Apache License 2.0, copyright oMLX contributors
- Runs entirely locally through Apple's MLX framework and mlx-lm's BatchGenerator β no cloud connection required
- Requires macOS 15.0 (Sequoia) or later, Apple Silicon (M1βM5), and Python 3.11β3.13; it does not support Intel Macs, Windows, or Linux
- Ships a native macOS menu-bar app (Swift/SwiftUI, not Electron) alongside a
omlxcommand-line tool and background server - Tiered key-value cache β hot blocks in RAM, cold blocks offloaded to SSD in safetensors format, persisting across server restarts with prefix sharing and copy-on-write
- Exposes OpenAI-compatible (
/v1/chat/completions,/v1/completions,/v1/embeddings,/v1/rerank) and Anthropic-compatible (/v1/messages) endpoints onhttp://localhost:8000 - One-click setup for coding-agent integrations β including Claude Code, Cursor, OpenClaw, OpenCode, Codex, Hermes Agent, Copilot, and Pi β from its built-in admin dashboard
- Built by a single developer, jundot, first announced publicly on an MLX GitHub discussion thread in March 2026; canonical repository is github.com/jundot/omlx β several same-named forks exist under other GitHub usernames and are not the original project
π In One Sentence
oMLX is a free, open-source local inference server for Apple Silicon Macs that runs models through MLX, ships as a menu-bar app plus a CLI/server, and adds a paged SSD cache so coding-agent contexts reload fast instead of recomputing from scratch.
π¬ In Plain Terms
Instead of a model reloading its entire conversation history from zero every time you reopen a coding agent, oMLX saves the processed version of that context to your Mac's SSD and restores it in seconds. It runs entirely on your own Mac, has a menu-bar icon so you don't need to touch the terminal, and also speaks the same API formats as OpenAI and Anthropic so tools like Claude Code or Cursor can point at it directly.
πNote: This review is the deep-dive companion to oMLX's entry in the Local LLM Software Directory β see that page for how oMLX compares at a glance to dozens of other local AI tools.
What Is oMLX?
oMLX is a local inference server for Apple Silicon Macs that turns Apple's MLX machine-learning framework into a full serving stack β an OpenAI/Anthropic-compatible API, a multi-model manager, a paged SSD cache, and a native macOS menu-bar app to control it all, without requiring a terminal for day-to-day use. Its own GitHub description positions it as "an LLM inference server optimized for Apple Silicon Macs, featuring continuous batching, tiered KV caching, and a native macOS menu bar management interface."
- Core function: a background inference server that loads open-weight models and serves them over a local HTTP API, managed either from a menu-bar app or the
omlxCLI - Inference engine: built on Apple's MLX framework and the mlx-lm library's
BatchGenerator, per oMLX's own documentation, with an original continuous-batching and paged-cache layer built on top - Foundation: per the developer's own announcement, oMLX was originally built on top of vllm-mlx as a starting point, with original implementations added for SSD cache tiering, continuous batching, vision-language model support, the Anthropic-compatible API, and the native macOS interface
- Developer: jundot (contact listed as junkim.dot@gmail.com in the repository), an independent maintainer β this review found no evidence of a company or funding round behind oMLX
- Canonical repository: github.com/jundot/omlx β several identically-named forks exist under other GitHub usernames (for example mkmsyk/omlx, dannysl/omlx); those are not the original project
oMLX's Origin and Growth
oMLX was first announced publicly by its developer, jundot, in a March 2026 post on Apple MLX's own GitHub discussions board. The announcement framed oMLX as a fix for a specific, concrete problem: coding agents that reuse a long, growing context on Apple Silicon could see multi-tens-of-seconds delays reloading that context on every request, because MLX-based servers of the time typically kept key-value cache state only in RAM and lost it whenever a server restarted or a context aged out. oMLX's paged SSD cache persists that state to disk instead, so a previously processed prefix can be restored from SSD rather than recomputed from the model. According to the developer's own announcement, at launch oMLX had already accumulated roughly 110 GitHub stars, built on top of vllm-mlx as a starting point with newly written code for SSD tiering, continuous batching, vision-language model support, an Anthropic-compatible API, and the native macOS interface. As of this review's publish date, oMLX's GitHub repository shows roughly 21,859 stars β fast growth for a project that was six months old at the time of writing.
- Announcement: March 4, 2026, on github.com/ml-explore/mlx/discussions/3203, posted by the project's own developer, jundot
- Starting point: vllm-mlx, with the developer's own SSD-tiering, continuous-batching, VLM, Anthropic-API, and native-UI code added on top
- Growth: from roughly 110 GitHub stars at the March 2026 announcement to roughly 21,859 stars as of this review, per oMLX's own GitHub repository
- License history: the repository's LICENSE file carries a 2025 copyright notice for "oMLX contributors," ahead of the project's public March 2026 announcement β consistent with private development before the public launch
What Can You Do With oMLX?
oMLX's feature set centers on turning MLX model inference into a persistent, multi-model local server, rather than a one-off script. Here is what each part actually does, per oMLX's own GitHub README and omlx.ai documentation.
- Tiered key-value cache β active context blocks stay in a hot RAM tier; inactive blocks move to a cold SSD tier in safetensors format, with prefix sharing and copy-on-write, and the cache survives server restarts instead of being discarded
- Continuous batching β concurrent requests are handled through mlx-lm's
BatchGenerator, with a configurable maximum concurrency, rather than processing one request at a time - Multi-model serving β load LLMs, vision-language models (VLMs), embedding models, and rerankers side by side, with LRU eviction, manual load/unload, model pinning, and a configurable per-model time-to-live
- OpenAI- and Anthropic-compatible API β
/v1/chat/completions,/v1/completions,/v1/embeddings, and/v1/rerankfor OpenAI-format clients, plus/v1/messagesfor Anthropic-format clients, all served fromhttp://localhost:8000 - Vision-language and OCR support β multi-image chat with base64, URL, or file inputs, tool calling with vision context, and auto-detection of dedicated OCR models
- Tool calling and MCP β JSON-schema tool calling with automatic chat-template detection for Llama, Qwen, DeepSeek, Gemma, GLM, MiniMax, Mistral, and other model families, plus Model Context Protocol (MCP) configuration
- Native macOS menu-bar app β a Swift/SwiftUI app (explicitly not Electron, per the project's own documentation) for starting, stopping, and monitoring the server without a terminal, with local usage analytics (per-model totals, an hourly usage heatmap) and automatic restart on crash
- Admin dashboard β a web UI at
/admin, reachable from a browser once the server is running, for model management, a built-in chat interface, one-click benchmarking, a model downloader, and one-click setup for coding-agent integrations including Claude Code, Cursor, OpenClaw, OpenCode, Codex, Hermes Agent, Copilot, and Pi - Experimental distributed inference β multi-Mac inference across MLX pipeline ranks over Ring or Thunderbolt RDMA networking, marked experimental in oMLX's own documentation
Usage Examples: Three Ways to Use oMLX
These are concrete workflows built from oMLX's documented features above β not hypothetical use cases.
Install oMLX: macOS App, Homebrew, and CLI
oMLX installs as a macOS app, via Homebrew, or from source β there is no Windows or Linux build, because it depends on Apple's MLX framework, which only targets Apple Silicon. All three methods install the same underlying omlx command-line tool. Links below are from the official GitHub README β always verify against that page directly, since install instructions can change between releases.
macOS app (.dmg)
- Command / Link:
- Download from GitHub Releases, drag to Applications. Includes auto-update and installs a CLI shim at
~/.omlx/bin/omlx.
Homebrew
- Command / Link:
brew tap jundot/omlx https://github.com/jundot/omlx, thenbrew install jundot/omlx/omlx, thenomlx start
From source
- Command / Link:
git clone https://github.com/jundot/omlx.git, thencd omlx, thenpip install -e .
Run the server directly
- Command / Link:
omlx serve --model-dir ~/models(foreground) oromlx start/omlx stop/omlx restart(background)
System requirements per the official README: macOS 15.0 (Sequoia) or later, Apple Silicon (M1 through M5), and Python 3.11β3.13. There is no Intel Mac, Windows, or Linux build. A source-install variant adds OMLX_WITH_CUSTOM_KERNEL=1 pip install -e . for specific model families (GLM-5.2/MiniMax M3, per the README) that benefit from a custom compiled kernel. Server configuration flags β --memory-guard {safe|balanced|aggressive}, --memory-guard-gb, --paged-ssd-cache-dir, --hot-cache-max-size, --max-concurrent-requests, --mcp-config, --hf-endpoint, --api-key, and --host β persist to ~/.omlx/settings.json once set, and can be overridden per-run with CLI flags.
oMLX Pricing: Is oMLX Really Free?
Yes β oMLX has no paid tier. Neither the GitHub repository nor omlx.ai has a pricing page, and the LICENSE file is a plain, unmodified Apache License 2.0 with a 2025 copyright notice for "oMLX contributors" β it applies to the entire application, with no clause gating any feature behind payment.
- No subscription, no paid tier, no usage limits imposed by oMLX itself
- No account or sign-up required to install or run the app
- The application, the menu-bar app, the CLI, and the admin dashboard are all covered by the same Apache License 2.0 terms
- Because oMLX only runs local models through MLX, there is no per-token or per-request cost from a cloud provider either β the only cost is the electricity and hardware you already have
oMLX vs. mlx-lm
mlx-lm is the Python library and simple CLI that Apple's MLX ecosystem provides for running and serving models β and oMLX is built directly on top of it, using its BatchGenerator for continuous batching. The two are not really competitors so much as different layers: mlx-lm is the foundation, oMLX is a full serving product built on that foundation.
Aspect | oMLX | mlx-lm |
|---|---|---|
| What it is | A packaged inference server: menu-bar app + CLI + admin dashboard | A Python library and lightweight CLI for running/serving MLX models |
| KV caching | Tiered RAM + SSD paged cache, persists across restarts | In-memory cache only, per its own documented server; no SSD tiering |
| Multi-model management | LRU eviction, pinning, per-model TTL, load/unload from a UI | Not a built-in feature β scripted or managed manually |
| API compatibility | OpenAI-compatible and Anthropic-compatible endpoints | OpenAI-compatible server mode (mlx_lm.server) |
| Interface | Native macOS menu-bar app plus web admin dashboard | Command-line only, no GUI |
| Coding-agent integrations | One-click setup for Claude Code, Cursor, OpenClaw, and others | Not a documented feature; requires manual endpoint configuration |
If you want the simplest possible way to run one MLX model from a script or a quick server command, mlx-lm alone may be all you need. If you want a persistent, multi-model server with a menu-bar UI, SSD-backed caching for long agent contexts, and one-click coding-agent integrations, oMLX is built specifically to add that layer on top of mlx-lm rather than replace it.
Who Should Use oMLX?
Whether oMLX fits depends almost entirely on one thing first: whether you are on an Apple Silicon Mac at all, since that requirement has no workaround.
oMLX vs. Other Local Inference Tools
oMLX sits in the Apple Silicon / MLX corner of the local inference landscape. Here is how it compares to other tools in that same segment β see the Local LLM Software Directory for the full catalog, and the dedicated oMLX vs. mlx-lm comparison above for the closest head-to-head.
- mlx-lm β the underlying Python library and CLI that oMLX itself is built on top of; the right choice if you want the simplest possible way to run one MLX model without a full server layer. See the dedicated comparison section above.
- exo β an open-source tool for clustering multiple Apple Silicon Macs (and other devices) together to run larger models than any single machine could hold; relevant if oMLX's experimental single-cluster distributed mode isn't enough for your setup. See the exo review.
- LM Studio β a cross-platform (macOS, Windows, Linux) desktop app that also uses MLX as one of its inference engines on Apple Silicon, alongside llama.cpp; a better fit if you need the same tool to work on non-Mac hardware too. See the LM Studio review.
- llamafile β a single-executable local inference approach built on llama.cpp rather than MLX, that runs the same file across macOS, Windows, and Linux with no install step; a useful contrast if cross-platform portability matters more to you than Apple Silicon-specific performance. See the llamafile explained article.
Common Mistakes When Evaluating oMLX
Most confusion about oMLX comes from assuming it works like a cross-platform tool, confusing it with a same-named fork, or expecting Intel Mac support.
Frequently Asked Questions
What is oMLX?
oMLX (github.com/jundot/omlx) is a free, open-source local inference server for Apple Silicon Macs. It runs models through Apple's MLX framework and ships as a native macOS menu-bar app plus a omlx command-line tool and background server, with OpenAI- and Anthropic-compatible API endpoints.
Is oMLX free?
Yes. Neither the GitHub repository nor omlx.ai has a pricing page, and the LICENSE file is a plain, unmodified Apache License 2.0 with no paid tier of any kind.
Does oMLX run on Windows or Linux?
No. oMLX is macOS-only, and specifically requires Apple Silicon (M1 through M5) and macOS 15.0 (Sequoia) or later, because it depends on Apple's MLX framework, which does not target Windows, Linux, or Intel Macs.
Does oMLX run on an Intel Mac?
No. oMLX's documented system requirements specify Apple Silicon only. MLX, the framework it is built on, does not support Intel-based Macs.
How is oMLX different from mlx-lm?
mlx-lm is the underlying Python library and lightweight CLI that Apple's MLX ecosystem provides for running models; oMLX is built on top of mlx-lm's BatchGenerator and adds a persistent server, a menu-bar app, a tiered SSD-backed cache, multi-model management, and OpenAI/Anthropic-compatible endpoints. See the full comparison above.
What is the paged SSD cache in oMLX for?
It offloads inactive key-value cache blocks from RAM to your Mac's SSD, in safetensors format, instead of discarding them. When a long-running coding agent revisits a large context, oMLX can restore the previously processed blocks from SSD rather than recomputing them from the model, which its own documentation credits with cutting reload waits on long contexts from tens of seconds down to a few seconds.
Can I use oMLX with Claude Code or Cursor?
Yes. oMLX's admin dashboard includes one-click integration setup for Claude Code, Cursor, OpenClaw, OpenCode, Codex, Hermes Agent, Copilot, and Pi, and its API is compatible with both the OpenAI and Anthropic request formats those tools use.
Is oMLX open source? What license does it use?
Yes. oMLX's LICENSE file is a plain, unmodified Apache License 2.0, with a 2025 copyright notice for "oMLX contributors."
Who develops oMLX?
oMLX is developed by jundot, an independent maintainer (contact listed as junkim.dot@gmail.com in the repository). This review found no evidence of a company or funding round behind the project.
Is github.com/jundot/omlx the real oMLX repository?
Yes. Several identically-named "omlx" repositories exist under other GitHub usernames, but github.com/jundot/omlx is the original project, confirmed by the developer's own announcement on Apple MLX's GitHub discussions board.