Key Takeaways
- OlliteRT (github.com/NightMean/OlliteRT) is a free, open-source Android app that runs LLMs locally and serves them over HTTP β not a chat app
- Created by developer NightMean; repository created April 6, 2026
- Apache-2.0 licensed, confirmed via the GitHub repository's license metadata
- Runs entirely on-device via Google's LiteRT-LM runtime, using
.litertlmmodel files exclusively β it does not support GGUF - Exposes an OpenAI-compatible HTTP API (chat completions, completions) plus Anthropic Messages API-compatible endpoints, for other devices on the local network to call
- Over 350 GitHub stars and 47 forks as of this review; latest tagged release is v0.9.6-beta.1 (June 6, 2026), so the project has not yet reached a 1.0 release
π In One Sentence
OlliteRT is a free, open-source (Apache-2.0) Android app, created by developer NightMean, that turns a phone into a fully local, OpenAI-compatible LLM inference server using Google's LiteRT-LM runtime, with over 350 GitHub stars as of this review.
π¬ In Plain Terms
Instead of running a chat app on your phone, OlliteRT turns the phone itself into a small server: you load a model onto it, start the server, and other devices on your home network β a laptop, a script, a home-automation tool β can send it text and get a response back, the same way they would talk to OpenAI's API, except everything runs on the phone.
πNote: This review is based on OlliteRT's own GitHub repository, README, and release history. It does not claim PromptQuorum has run hands-on throughput or battery-drain benchmarks on a specific phone model.
What Is OlliteRT?
OlliteRT is an Android application that turns the phone it runs on into a local LLM inference server, described in its own GitHub repository description as letting you "turn your Android phone into an OpenAI-compatible LLM inference server β fully local, private and Open Source." It has no built-in chat window of its own; its purpose is to run inference and answer HTTP requests from other software.
- Product type: an Android backend server application, not a chat client β no chat UI is included
- Creator: developer NightMean; the GitHub repository is hosted at github.com/NightMean/OlliteRT
- Repository created April 6, 2026, per GitHub's repository metadata β a young project as of this review
- License: Apache-2.0, confirmed via the GitHub repository's license metadata
- Inference engine: Google's LiteRT-LM on-device runtime, the same underlying technology Google uses for on-device AI features in its own Android tooling
- Scale: over 350 GitHub stars and 47 forks as of this review
Project History and Version Milestones
OlliteRT's GitHub repository was created on April 6, 2026, and it has shipped three tagged pre-1.0 releases since, all still labeled beta.
- 1April 6, 2026: Repository created
Why it matters: Marks the start of the project, per GitHub's repository metadata. - 2v0.9.0-beta.1 β April 24, 2026
Why it matters: The first tagged release found on the project's GitHub releases page. - 3v0.9.5-beta.1 β April 30, 2026
Why it matters: A follow-up beta release roughly a week after the first tag. - 4v0.9.6-beta.1 β June 6, 2026
Why it matters: The most recent tagged release as of this review; the project has not yet shipped a 1.0.
What Does OlliteRT Actually Do?
OlliteRT loads a .litertlm model file on the phone, runs inference through Google's LiteRT-LM runtime using the phone's own CPU and GPU, and serves the result over an OpenAI-compatible HTTP API that other devices on the same network can call.
- On-device inference: uses Google's LiteRT-LM runtime exclusively β no cloud fallback, no other backend engine
- Model format:
.litertlmfiles only; it does not support GGUF, the format used by llama.cpp-based tools - Model acquisition: one-tap downloads directly from HuggingFace for supported models, per the project README
- API compatibility: exposes OpenAI-compatible chat-completions and completions endpoints, plus Anthropic Messages API-compatible endpoints, over HTTP on the local network
- Multimodal support: vision, audio, and "thinking" (reasoning) models are supported where the underlying model itself supports those modes, per the README
- Hardware acceleration: configurable GPU or CPU acceleration settings per model
- Operational tooling: a built-in benchmarking tool, activity logging with JSON highlighting, a server-monitoring dashboard, Prometheus metrics integration, and a Home Assistant REST API integration
- Single-model, sequential design: the README documents that only one model is loaded at a time and requests are processed sequentially, not in parallel
Usage Examples: Two Ways to Use OlliteRT
These are concrete workflows built from the project's documented features above.
Install OlliteRT
OlliteRT installs free as an APK from GitHub releases, and its source code is on GitHub.
Source | Link |
|---|---|
| GitHub releases (APK download) | github.com/NightMean/OlliteRT/releases |
| GitHub repository (source code, Apache-2.0) | github.com/NightMean/OlliteRT |
As of this review, OlliteRT is distributed only via direct APK download from GitHub releases β this review found no listing on Google Play or F-Droid. Requires Android 12 or later on an arm64-v8a device. Installing an APK from outside Google Play requires enabling "install from unknown sources" in Android settings; only download the APK from the official GitHub releases page.
Platform, Pricing, and Licensing
Platform
- What OlliteRT states:
- Android only (arm64-v8a devices, Android 12+). No iOS, desktop, or web version.
Cost
- What OlliteRT states:
- Free and open source. No account, subscription, or in-app purchase found in the README.
Licensing
- What OlliteRT states:
- Apache-2.0, confirmed via the GitHub repository's license metadata.
Hardware minimum
- What OlliteRT states:
- Minimum 6 GB RAM per the README; 8 GB+ recommended, and more for larger or multimodal models such as Gemma 4 E4B (needs 12 GB RAM).
Install method
- What OlliteRT states:
- Direct APK download from the GitHub releases page; no Google Play or F-Droid listing found as of this review.
Verify current hardware recommendations and supported model list directly on GitHub before choosing a phone or model, since both can change as the project (still pre-1.0) matures.
OlliteRT vs. PocketPal AI
OlliteRT and PocketPal AI both run LLMs locally on a phone, but for different purposes: OlliteRT is a headless server other devices talk to, while PocketPal AI is a chat app you use directly on the phone itself.
Aspect | OlliteRT | PocketPal AI |
|---|---|---|
| Primary use | Headless inference server for other devices on the network | On-device chat app used directly on the phone |
| Chat interface | None included | Built-in chat UI |
| Inference engine | Google LiteRT-LM | llama.cpp-based (GGUF models) |
| Model format | .litertlm only | GGUF |
| Platform | Android only | Android and iOS |
| API server | OpenAI-compatible HTTP API, core feature | Not a core feature |
If you want to send requests from a laptop, script, or smart-home hub to a phone acting as a server, OlliteRT is built for that. If you want to chat with a local model directly on the phone's own screen, a chat-first app like PocketPal AI fits better β see the PocketPal AI review for details.
Who Should Use OlliteRT?
OlliteRT fits developers and hobbyists who want to repurpose an Android phone as a dedicated, fully local LLM server for other devices, rather than chat with a model on the phone itself.
What OlliteRT Is Not Good For
OlliteRT is not a good fit if you need a chat interface, GGUF model support, parallel request handling, or a long-established, stable release history.
- Not a chat app β it has no built-in chat window; it only answers HTTP API requests from other software
- Not GGUF-compatible β it works exclusively with
.litertlmmodel files, so it cannot load the GGUF models used by llama.cpp, Ollama, or most other local-LLM tools without a separate conversion step - Not built for concurrent load β the README documents a single loaded model processed sequentially, so it is not designed as a multi-user or high-throughput server
- Not iOS-compatible β Android only, and specifically arm64-v8a devices
- Not a mature 1.0 release β the repository is roughly five months old as of this review, and its latest tagged version (v0.9.6-beta.1) still carries a beta label
Common Mistakes When Evaluating OlliteRT
Most confusion about OlliteRT comes from assuming it is a chat app, confusing its model format with GGUF, or missing its young project status.
Competitors and Alternatives
OlliteRT is most often compared to other Android/mobile local-LLM apps β its main differentiator is being a headless, OpenAI-compatible server rather than an on-phone chat client.
Tool | Best known for | Link |
|---|---|---|
| PocketPal AI | On-device chat app for Android and iOS, GGUF models via llama.cpp | PocketPal AI review |
| Layla | Android chat app focused on local, uncensored roleplay and assistant use | Layla review |
| Google AI Edge Gallery | Google's own showcase app for on-device models via LiteRT/MediaPipe | Google AI Edge Gallery review |
| MLC Chat | Cross-platform (Android/iOS) local chat app built on the MLC LLM compiler stack | MLC Chat review |
This list reflects tools commonly compared to OlliteRT in the mobile/on-device LLM space, not an independent PromptQuorum ranking β verify each tool's current feature set and supported model formats before choosing.
Frequently Asked Questions
What is OlliteRT?
OlliteRT (github.com/NightMean/OlliteRT) is a free, open-source (Apache-2.0) Android app, created by developer NightMean, that turns a phone into a fully local, OpenAI-compatible LLM inference server using Google's LiteRT-LM runtime.
Does OlliteRT have a chat interface?
No. OlliteRT has no built-in chat window. It is a headless server β you interact with it via its HTTP API from another device or application, not by typing directly into the app.
Is OlliteRT free?
Yes, OlliteRT is free and open source under the Apache-2.0 license. This review found no account, subscription, or in-app purchase requirement.
What model format does OlliteRT use?
OlliteRT uses .litertlm model files exclusively, for Google's LiteRT-LM on-device runtime. It does not support GGUF, the format used by llama.cpp-based tools.
What are the hardware requirements for OlliteRT?
Android 12 or later on an arm64-v8a device, with a minimum of 6 GB RAM (8 GB+ recommended). Larger or multimodal models, such as Gemma 4 E4B, need 12 GB RAM or more, per the project README.
Does OlliteRT work without an internet connection?
Inference itself runs fully on-device once a model is downloaded, so no internet connection is required for that. A local network connection is needed for other devices to reach the phone's API server, and internet access is needed to initially download models from HuggingFace.
Can OlliteRT handle multiple requests at once?
No. The project README documents that only one model is loaded at a time and requests are processed sequentially, not in parallel β it is not designed for concurrent, multi-user load.
Is OlliteRT available on iOS?
No, OlliteRT is Android-only, and specifically requires an arm64-v8a device running Android 12 or later.
How do I install OlliteRT?
Download the APK directly from the GitHub releases page. As of this review, it is not listed on Google Play or F-Droid.
Is OlliteRT a mature, stable project?
Not yet. Its GitHub repository was created in April 2026, and its latest tagged release (v0.9.6-beta.1, June 2026) still carries a beta label β treat it as an actively developing, pre-1.0 project.