Skip to main content
PromptQuorumBuilt for humans. Structured for AI.
Home/Power Local LLM
Mobile & Edge LLMs

Local AI on Android in 2026: What Is Possible, What Is Not, and the Apps We Cover

·45 min readBy · Founder of PromptQuorum · Discovery engine for open-weight & open-source AI

Yes: a recent Android phone can run a useful local AI offline, but only small models. Expect 1 to 4 billion parameters on 6 to 8 GB of RAM and 7 to 8 billion at low-bit quantization on 12 to 16 GB. Chat, camera questions, dictation, document Q&A and translation work; frontier-model quality, long contexts, sustained speed and an always-on background assistant do not. Android adds options that are broader or easier than on iPhone: installing apps outside the store, Termux, and running the phone as a local API server.

Key Takeaways

  • Local AI on Android is real and offline-capable, and it is small-model AI. Phones with 6 to 8 GB of RAM handle 1 to 4 billion parameter models; 12 to 16 GB phones can hold 7 to 8 billion at low-bit quantization. Cloud-class answers, long documents and fast long replies are out of reach.
  • RAM decides what fits; memory bandwidth, heat and the chip decide how fast. A model must fit in free RAM next to Android itself, and generation speed is capped by how fast the phone can read the model from memory on every token.
  • There are four engines behind almost every app: llama.cpp (GGUF files, CPU and sometimes GPU), Google LiteRT-LM (Gemma and other models, GPU and increasingly NPU), MLC (compiled models, OpenCL GPU) and Google AICore with Gemini Nano (system model, supported phones only, apps call it through ML Kit).
  • The NPU is the least reachable part of the phone. Google, Qualcomm and MediaTek publish NPU paths through LiteRT and Qualcomm GenieX, but they need models compiled per chip. A generic sideloaded GGUF app will usually run on CPU or GPU.
  • Android gives more room than iPhone in several ways: apps can be installed from outside Google Play, Termux gives you a Linux terminal on the phone, phones can serve an OpenAI-compatible API to your laptop, and GGUF model import is a normal feature. The price is fragmentation: results differ by brand, chip and battery settings.
  • Most Android apps in our directory are mapped to a job in this guide: chat, vision, voice, documents, translation, image generation, photos, notes, smart home and agents, developer tools, and remote clients for your own server.
  • "Zero telemetry" is a claim, not a guarantee. On-device inference keeps prompts off servers, but the app can still ship analytics, ads or crash reports. Read the Data safety card and the privacy policy together; they disagree more often than you would expect.
  • Not possible today: frontier-model quality offline, 70 billion parameter models, million-token contexts, an always-on background assistant that Android will not kill, and Gemini Nano access for any app on any phone.
Read in:
🇺🇸en🇩🇪de🇫🇷fr🇯🇵ja🇨🇳zh🇪🇸es🇧🇷pt🇸🇦ar🇰🇷ko

Android phones can run local AI offline in 2026: chat, camera questions, dictation, document Q&A and translation all work with models of roughly 1 to 4 billion parameters, and 7 to 8 billion on 12 to 16 GB phones. This guide maps the main capabilities, the hard limits, the chip and RAM reality, and links the Android apps in our directory to the job each does best.

Local AI on Android in 2026: What Is Possible, What Is Not, and the Apps We Cover

Start Here: Three Apps to Install

Short on time? Install one of these three, download a model that fits your phone's RAM, and test it once in airplane mode. The picks come from this guide's own chapters and reviews, not from benchmarks; the rest of the guide explains each choice.

Other good starting points: on a phone where AICore supports Gemini Nano, such as a Pixel 9 or 10, try PAIOS, which uses Google's system model with no download; for camera questions and Gemma models, use Google AI Edge Gallery; to run bigger models from your phone, connect it to Ollama or LM Studio on your own computer with LocalMind.

📍 In One Sentence

Beginners can start with PocketPal AI, advanced users with Maid and a larger GGUF model, and developers with the ML Kit Prompt API or LiteRT-LM; in every case the model size follows the phone's RAM.

💬 In Plain Terms

One app, one model sized to your RAM, one airplane-mode test.

Beginner

Install or use:
PocketPal AI
First model:
1 to 2B on 4 to 6 GB of RAM; 3 to 4B on 8 GB
Why this pick:
Free, MIT-licensed and no account needed. You choose the model, and an oversized model simply fails to load, so go one size down if it does.

Advanced

Install or use:
Maid
First model:
4 to 8B on 12 GB; 7 to 8B on 16 GB, with a short context
Why this pick:
A GGUF runner for the largest model your phone can hold. Expect a slower, hotter phone and keep the context short.

Developer

Install or use:
ML Kit Prompt API or LiteRT-LM
First model:
Google system model, or your own model
Why this pick:
The Prompt API covers small text tasks on the system model; LiteRT-LM is for your own model. The Developer Paths chapter compares the other routes.

Six Quick Answers Before You Install

These are the six questions readers ask first; each answer points to the chapter that explains it.

📍 In One Sentence

RAM picks the model size, 3 to 6 GB of free storage holds a first model, and an airplane-mode test shows whether the app works offline.

💬 In Plain Terms

RAM picks the model, storage holds it, and airplane mode proves it.

  • Which model should I download? Match your RAM: 1 to 2B on 4 to 6 GB, 3 to 4B on 8 GB and 4 to 8B on 12 to 16 GB, where 7 to 8B fits only with a short context. When in doubt, pick the smaller one. See RAM Tiers below.
  • How much storage do I need? Free 3 to 6 GB for a first model, because model files are measured in gigabytes. See First Run in 15 Minutes.
  • How fast will it run? PromptQuorum did not run new speed tests for this guide. Speed is limited by memory bandwidth, so a smaller model answers faster; note how quickly words appear on your first prompt and treat that as your baseline. See the Chips chapter.
  • Which app is private? None can be assumed private. On-device inference keeps your prompts off a vendor's inference servers, but an app can still send analytics. Check the Data safety card, the privacy policy and the permissions, and run the airplane-mode test; open-source apps such as PocketPal AI can be audited, while closed-source claims rest on trust. See Privacy and Telemetry Reality.
  • What can I do without internet? According to this guide's capability map, chat, camera questions, dictation, document questions and translation can work offline with suitable models after the download; image generation and large models are slow or need a server. See The Capability Map.
  • What do I read next? Follow First Run in 15 Minutes, use Summary: Phone Tier, App, Model Size as a one-table cheat sheet, and open the Decision Path if you have a specific job in mind.

Quick Facts

  • Practical model sizes: 1 to 2 billion parameters on 4 to 6 GB RAM; 3 to 4 billion on 8 GB; 7 to 8 billion at low-bit quantization on 12 to 16 GB. These are rules of thumb from file-size arithmetic, not benchmark results.
  • Gemini Nano via ML Kit GenAI: the Prompt API takes text and up to 10 images, with input under about 4,000 tokens, and works only while the calling app is in the foreground (Google).
  • Gemma 4 E2B and E4B: Google calls them the foundation of Gemini Nano 4 and says E2B is up to 3 times faster than E4B; the AICore Developer Preview opened on 2 April 2026 (Android Developers Blog).
  • NPU paths: LiteRT has Qualcomm and MediaTek NPU accelerators; Google reports Gemma 3n E2B at over 1,600 tokens/sec prefill and 28 tokens/sec decode on a Dimensity 9500 NPU (Vivo X300 Pro, Google figures, 4K context).
  • Pixel TPU for developers: the Tensor SDK reached beta on 19 May 2026 and lists Pixel 10, 10 Pro, 10 Pro XL and 10 Pro Fold.
  • Android versions: Google AI Edge Gallery needs Android 12 or later; the NPU docs for LiteRT require API level 31 or later; AppFunctions needs Android 16 or later.
  • Termux: install it from F-Droid or GitHub; the Google Play build is marked experimental by the Termux maintainers, and mixing sources breaks updates.
  • Apps covered: the Android apps in the PromptQuorum directory that we reviewed from public sources, each linked once to its own review where it fits; see the directory for the full list.
  • iPhone reader? See the sibling guide Local AI on iPhone.

The Capability Map: What Works, Partly Works, Does Not

Most everyday AI jobs work on an Android phone without a connection; the jobs that need scale or constant background activity do not. The table gives the verdict per capability, one app to start with, and the hard limit. Each chapter below lists the apps we cover for that job.

"Works" means small on-device models do the job acceptably today. "Partly" means it works with a server in the loop, with one app only, or with clear quality limits. "No" means the limit is physical or a platform rule, not a missing app.

📍 In One Sentence

On Android in 2026, chat, camera questions, dictation, translation and short-document Q&A run offline on small models, while large models, long contexts and always-on assistants do not.

💬 In Plain Terms

Think of an on-device model as a capable intern with a small desk: fast, private and offline, but not able to hold a whole library or match a cloud model on hard questions.

Capability
On the phone?
Start with
Hard limit
Offline chatWorksPocketPal AISmall models only
Camera and image questionsWorksGoogle AI Edge GalleryNeeds a vision-capable model
Dictation (speech to text)WorksPrivate MindAccuracy varies by language
Read-aloud repliesWorksToolNeuronVoice quality varies by app
Offline voice assistantPartlyHome Assistant AssistAndroid may kill background listeners
Document and PDF questionsWorksAnythingLLM MobileContext window of small models
TranslationWorksLLM HubWeaker on rare languages
Image generationPartlyTokForgeSlower than a desktop GPU
Photo searchPartlyEnte PhotosOften runs on a server or desktop
AI in notes and bookmarksPartlyKarakeepAI usually runs on your server
Smart-home hubPartlyScryptedPhone is a client, not the brain
Agents that drive the phoneEarlyKai 9000System hooks still in preview
Phone as a thin clientWorksLocalMindYour server must be on
Build AI into your own appWorksRunAnywhereNPU needs per-chip builds
70 billion parameter modelsNoUse a server insteadRAM
Cloud-class answers offlineNoUse a server insteadModel size

💡Tip: If a job says "Partly", the usual fix is the remote pattern: the phone is the screen and microphone, and a computer or home server runs the bigger model. See the remote mode chapter.

How This Guide Was Built

This is a desk-research map, not a benchmark report. It was written by Hans Kuepper from Google, Qualcomm, MediaTek and Termux documentation read in November 2026, and from the individual PromptQuorum reviews linked in each chapter. PromptQuorum did not run new speed tests for this guide.

Many of the linked reviews state that they rest on store listings, privacy policies and maker documentation and that PromptQuorum has not run the app. Where a claim comes from the maker or from Google, the text says so. Where we could not confirm something against a primary source, the text says "uncertain" or leaves it out.

Speed numbers appear only when a named source published them, with the device named. Model sizes, RAM tiers and the "tokens per second ceiling" are arithmetic, labeled as such.

📍 In One Sentence

This guide combines official platform documentation with PromptQuorum app reviews and runs no new benchmarks, so every number is either sourced or labeled as arithmetic.

💬 In Plain Terms

If a sentence names a speed, a device and a source, it was published by that source; if it has no source, it is a rule of thumb.

RAM Tiers: Which Model Size Your Phone Can Hold

RAM is the first filter: the model has to fit in free memory next to Android and your other apps. A 4-bit quantized model needs roughly 0.5 to 0.6 GB per billion parameters for the weights, plus working memory that grows with the context length. Android, the launcher and background services typically hold several gigabytes of a phone's RAM, so a 12 GB phone does not give an app 12 GB.

Speed follows memory bandwidth. Generating one token means reading most of the model from memory, so the ceiling is about bandwidth divided by model size. Our Galaxy S26 on-device AI guide applies this: 85.6 GB/s over a 3.5 GB 4-bit model gives a theoretical ceiling near 24 tokens per second. Real apps land below the ceiling, and a hot phone lands lower still.

Treat the table as a starting point. Several reviewed apps publish no fixed RAM floor and say the need depends on the model you load; others set their own limits. Ensu disables chat on phones with less than 4 GB of RAM, and Private Mind's Google Play listing recommends a phone with 4 GB or more for larger models.

📍 In One Sentence

A phone with 6 to 8 GB of RAM fits models of about 1 to 4 billion parameters at 4-bit quantization, and 12 to 16 GB phones can hold 7 to 8 billion.

💬 In Plain Terms

Quantization shrinks a model by storing each weight in fewer bits; a 4-bit model is roughly a quarter the size of a 16-bit one, at a small quality cost.

Phone RAM
Model size (4-bit)
Example models
What to expect
4 GB0.5 to 1BQwen3 0.6B, Gemma 3 1BShort answers, simple tasks
6 GB1 to 2BQwen3 1.7B, SmolLM2 1.7BUsable chat, basic summaries
8 GB3 to 4BPhi-4 Mini, Qwen3 4BBest balance on most phones
12 GB4 to 8B (tight)Gemma 4 E4B, Llama 3.2 3B7 to 8B only with small context
16 GB and up7 to 8B7 to 8B instruct modelsSlower, hotter, longer waits

⚠️Warning: Google's own pages publish no RAM floor for Gemma 4 E2B or E4B in the sources we read. Treat RAM needs as uncertain and check the app's model list on your phone.

💡Tip: When a model will not load, the cause is usually free RAM, not total RAM. Close other apps, restart the phone, and pick the next smaller model before blaming the app.

Chips: Snapdragon, Tensor, Dimensity, Exynos

Current flagship chips include an NPU, but a sideloaded chat app rarely gets to use it. The NPU paths that exist are published by the chip makers and Google, and they need models prepared for that chip. Apps built on generic llama.cpp builds mostly run on the CPU, sometimes on the GPU.

Snapdragon. Qualcomm's GenieX supports Android on the Snapdragon 8 Elite and 8 Elite Gen 5, runs GGUF models through llama.cpp with a Hexagon backend, and recommends Q4_0 quantization for the NPU path; the page does not state a release status. Google's LiteRT Qualcomm accelerator reports FastVLM-0.5B on the Snapdragon 8 Elite Gen 5 NPU at 0.12 seconds to first token and over 100 tokens per second decode (Xiaomi 17 Pro Max, Google figures). These are developer paths today, not buttons in a chat app.

Google Tensor. Pixel 9 runs Gemini Nano through AICore and ML Kit; the Pixel 10 with Tensor G5 is listed for Gemini Nano 4 support. For your own models, Google's Tensor SDK reached beta on 19 May 2026 for the Pixel 10 family and runs models on the TPU through LiteRT. Whether a given third-party chat app already uses it is something to check per app.

MediaTek Dimensity. Google and MediaTek published a LiteRT NeuroPilot accelerator with pre-compiled models (Qwen3 0.6B, Gemma 3 270M, Gemma 3 1B, Gemma 3n E2B, EmbeddingGemma 300M). The benchmark chip is the Dimensity 9500 in a Vivo X300 Pro, where Google reports Gemma 3n E2B at over 1,600 tokens per second prefill and 28 tokens per second decode with a 4K context. Converting a custom Gemma model for the MediaTek NPU was listed as "soon".

Samsung Exynos. In our Galaxy S26 article, the S26 and S26+ use the Exynos 2600 in Europe, Korea and most other markets and a Snapdragon in the US, China and Japan, while the S26 Ultra uses the Snapdragon 8 Elite Gen 5 everywhere. We found no primary documentation of an open NPU path for third-party language-model apps on Exynos, so assume CPU or GPU for sideloaded apps. Samsung's own EdgeFusion image generation on the Exynos 2600 NPU is a Samsung and Nota AI pipeline, not something a generic app inherits.

📍 In One Sentence

Snapdragon, Tensor and Dimensity all have published NPU or TPU routes through LiteRT or vendor SDKs, but a generic sideloaded GGUF app usually runs on CPU or GPU, and Exynos has no documented open path for third-party language models.

💬 In Plain Terms

The NPU is a fast lane that needs a model rebuilt for each chip; apps that ship one universal model file cannot use it.

Chip family
Open NPU or TPU path
Where it is documented
Status
Snapdragon 8 Elite / Gen 5Yes, per-chip buildsLiteRT, Qualcomm GenieXDeveloper paths
Tensor G5 (Pixel 10)Yes, via Tensor SDKGoogle Developers BlogBeta (May 2026)
Tensor G4 (Pixel 9)Gemini Nano via ML KitML Kit GenAISupported device
Dimensity 9500Yes, NeuroPilotGoogle Developers BlogPre-compiled models
Exynos 2600Not documented for LLMsNone foundAssume CPU or GPU
Older or mid-range chipsRarelyVariesCPU, small models

⚠️Warning: Do not pick a phone for local AI from an NPU "TOPS" number. Peak NPU throughput says little about a chat app that never uses the NPU. RAM size, memory bandwidth and cooling decide most real results.

The Four Main Engines Behind Android AI Apps

Nearly every app in this guide is a front end for one of four engines, and the engine explains the app's strengths and limits better than its marketing does. If you know the engine, you know which model files work, whether the NPU can be used, and where your data goes.

📍 In One Sentence

Most Android on-device AI apps run on one of four main engines: llama.cpp for GGUF files, LiteRT-LM for Gemma-class models, MLC for compiled models, or Google AICore for the system Gemini Nano model.

💬 In Plain Terms

The engine decides which model files you can load and how much of the chip gets used; the app decides how pleasant it is to use.

llama.cpp

Runs:
GGUF files from Hugging Face; the widest model choice, and most apps let you import your own file.
Acceleration:
CPU by default; some apps add GPU. A Hexagon backend exists in Qualcomm GenieX.
Example apps:
PocketPal AI, Maid, LM Playground, KernelAI
Articles about llama.cpp (9)

+180 more not shown

LiteRT-LM

Runs:
Google's edge runtime for Gemma and other models in .litertlm files, with vision and audio input and function calling. Google lists Gemma 4, Gemma 3n, Llama, Phi-4 and Qwen families.
Acceleration:
GPU and NPU paths; Python, Kotlin and C++ APIs are marked stable.
Example apps:
Google AI Edge Gallery, OlliteRT, MemoryAI

MLC LLM

Runs:
Models compiled ahead of time for one target; you get a curated library, not any file.
Acceleration:
OpenCL GPU on Android; support is reported to be uneven across phones (see the MLC Chat review).
Example apps:
MLC Chat, MLC LLM
Articles about MLC LLM (7)

Also mentioned in:

AICore with Gemini Nano

Runs:
Google's system model, managed and updated by Android; apps reach it through ML Kit GenAI, so there is nothing to download inside the app.
Acceleration:
Whatever the supported phone provides; apps cannot pick the hardware.
Example apps:
PAIOS

💡Tip: Other engines exist. Private Mind runs on the React Native ExecuTorch library, and Ensu and Atomic Chat bring their own stacks. The four above cover most of what you will install.

Where Android Gives You More Room Than iPhone

Android is the more open platform for local AI in the sense that apps can load their own models and engines: you can install apps outside the store, run a Linux userland, and turn the phone into a server. The cost is fragmentation. A recipe that works on a Galaxy flagship may fail on a mid-range phone from another brand, and no single vendor-neutral compatibility list exists that we found.

For the Apple side of every comparison below, read Local AI on iPhone. For the two big ecosystems side by side, see Galaxy vs iPhone On-Device AI.

📍 In One Sentence

Android allows sideloaded APKs, Termux, F-Droid and phone-hosted API servers more freely than iOS, at the price of uneven results across brands and chips.

💬 In Plain Terms

Android lets you tinker deeper; iPhone gives you more predictable results on fewer devices.

  • Sideloading and alternative stores. MLC Chat ships as an APK from the maker's GitHub releases, and the Google AI Edge Gallery offers an APK for people without Google Play. The same app can exist in several places, so keep to one source per app.
  • Termux. A real terminal on the phone, installable from F-Droid or GitHub, can build and run llama.cpp without root. It is also where Android's background limits bite hardest (see the battery chapter).
  • The phone as a server. OlliteRT, Mobile LM Server, Ollama Local AI and ToolNeuron expose an OpenAI-style endpoint so other devices on your network can use the phone's model.
  • Model choice. GGUF import is a normal feature in Android apps, so you can load almost any small open-weight model a developer publishes, not only a curated list.
  • Hardware variety. Gaming phones with 16 GB or more of RAM and active cooling can give more headroom than the iPhones we compare against. The flip side is that cheap phones with 4 GB are common, and they are poor local AI machines.
  • Where iPhone is simpler. One vendor, one memory budget per model class and fewer battery-optimization surprises make iPhone results easier to predict. If predictability matters more than flexibility, the sibling guide may fit you better.

Chat Assistants in Our Directory, by Type

Offline chat is the most widely supported job on Android in our directory, and the choice of app is mostly a choice of trade-offs: open source or closed, any GGUF file or a curated list, free or paid. Each line below says what the app does and the honest limit. Apps that fit a more specific job appear again in that job's chapter.

For a speed and setup comparison of the best-known apps, read Best Local LLM Apps for Android. For model choice, read Mobile LLM Models: Phi-4, Gemma and SmolLM and the Mobile Local LLMs overview. Tablet owners should add Run AI on a Tablet.

📍 In One Sentence

Dozens of Android apps run chat models on the phone; open-source GGUF runners give the widest model choice, closed-source apps give guided downloads, and Google AI Edge Gallery is Google's own front end for Gemma-class models.

💬 In Plain Terms

Pick an open-source runner if you want to choose models yourself, and a guided app if you want it to work in five minutes.

Open-source and maker-built runners (start here)

  • PocketPal AI: free, MIT-licensed GGUF runner for Android and iPhone, no account needed; an optional community marketplace sells premium presets. Limit: you pick the model, and the app has no fixed RAM floor, so an oversized model simply fails to load.
  • Google AI Edge Gallery: Google's free showcase for on-device models on LiteRT, with chat, image questions, audio transcription and prompt testing. It needs Android 12 or later and is installed from Google Play or as an APK. Limit: Google labels the app an experimental beta. Its App Store privacy label lists some device diagnostics (per our review of the iPhone build); check the Android Data safety card before treating it as zero-telemetry.
  • Maid: MIT-licensed; runs GGUF models through llama.cpp or connects to remote providers with your own key. The codebase was rewritten in React Native for version 3.0.0 and ships as an Android-only app today. Limit: the older iOS and desktop builds are gone, so old tutorials will not match.
  • Off Grid AI: free, MIT-licensed, account-free, for Android, iOS, macOS and Windows. Limit: the download figures on the maker's site are self-reported, and could not be cross-checked against the app stores.
  • Private Mind: MIT-licensed, from Software Mansion, running models through React Native ExecuTorch, with document Q&A, Whisper dictation and built-in speed benchmarks. Limit: since version 1.3.0 an optional Web toggle fetches pages from the internet, so keep it off for strict offline use.
  • Atomic Chat: Apache-2.0; browses over 1,000 Hugging Face models, exposes a local OpenAI-compatible API and supports agents. Limit: optional cloud providers are built in, so confirm none is switched on if you want offline only.
  • Ensu: AGPL-3.0, from the Ente Labs team, with image input, local voice dictation and no account. Limit: chat is disabled on phones with less than 4 GB of RAM, and PromptQuorum has not tested it hands-on.
  • Llamatik: MIT-licensed offline chatbot that doubles as the showcase for the developer's Kotlin Multiplatform SDK; the Android version reached Google Play in November 2025. Limit: one independent developer and a young app.
  • LM Playground: MIT-licensed GGUF runner with document Q&A, photo input for vision models, reasoning display and optional tools. Limit: independent developer; the review rests on the store listing, website and repository.
  • MLC Chat: Apache-2.0 app on the MLC compiler, with GPU-accelerated inference. It is an APK from the maker's GitHub releases, not a Google Play listing. Limit: the curated model library, uneven OpenCL support across phones, and a long gap since its last update (see the quiet apps chapter).

Free consumer apps, closed source

  • KernelAI: downloads open models (53, from 112 MB to 3.11 GB per the maker) and runs them through llama.cpp, with documents and vision. Limit: closed source from a single publisher, so claims rest on its own pages.
  • MemoryAI: runs models through llama.cpp and Google LiteRT, with document Q&A, an editable Agent Memory, hands-free voice and a PIN-locked Kid Mode. Limit: closed source, one independent developer.
  • Loci AI: picks llama.cpp or MLX per model, manages phone temperature and syncs memory across chats; can link to a desktop for bigger models. Limit: closed source; the "private" promise can depend on the apps staying offline after setup.
  • On Device LLM: free offline chat with GGUF import and an optional local OpenAI-compatible API. Limit: a new app, closed source; Google Play titles it "Local LLM", and it is not the iPhone apps with similar names.
  • LocalAI: Offline AI Chat LLM: llama.cpp chat with an optional premium tier, document chat, vision and structured output. Limit: closed source and no public repository found.
  • Chatty-mini: portrait-first GGUF chat from an indie developer. Limit: sparse public documentation, and the license is unconfirmed.
  • Caelum: free chat and document assistant that says it runs on the phone, with optional web search through Brave. Limit: no update since August 2025; see the quiet apps chapter.

Companions, paid apps and company-backed apps

  • Layla: a paid app (one-time purchase plus in-app tiers) built around a persona-and-roleplay companion running locally. Limit: closed source and a price tag; it is a companion first, a utility second.
  • Laria: paid Android assistant with downloadable on-device models, custom AI identities, image tools and optional cloud providers. Limit: closed source, and, per our review, the developer's page shows explicit sample content; read the age-rating notes first.
  • Secret AI: custom characters, Skills and MCP tools, running GGUF, MNN and MLX models. Limit: closed source; other products share the name, so check the publisher before installing.
  • CompactifAI APP: from Multiverse Computing, offers compressed models for offline use and larger models through the maker's cloud API. Limit: the cloud path is part of the product, so offline is an option you choose.
  • Liquid Apollo: Liquid AI's app for its small LFM2 models on the device, plus OpenRouter and custom backends. Limit: not local-only; the cloud and custom-server modes send prompts off the phone.
  • TokForge: roleplay characters, image generation, voice and document Q&A, with a speed benchmark that can post to a public leaderboard. Limit: proprietary, and the iPhone version is a public beta.

The system model: Gemini Nano through AICore

  • PAIOS: a free, open-source chat front end for Gemini Nano, running through AICore with no model download inside the app. Limit: it works only on phones where AICore supports Gemini Nano, with Pixel 9 and 10 as named examples, and the project labels itself alpha.

💡Tip: Install two apps, not ten: one open-source GGUF runner for choice (PocketPal AI or Maid) and Google AI Edge Gallery for Gemma-family models. Add others only for a job they do better.

⚠️Warning: Several apps above are new, closed source and reviewed from listings only. Their feature lists are the makers' claims; PromptQuorum's reviews say which were not tested hands-on.

Vision and Camera Questions

Pointing the camera at something and asking a question works on-device today, but only with a vision-capable model. A text-only model silently ignores or rejects images. The apps below load a vision model or one that accepts image input; the model, not the app, sets the quality.

Google's ML Kit offers system-level options too: the Prompt API accepts text plus up to 10 images, and a dedicated Image Description API produces a short caption of an image. Both run on Gemini Nano on supported phones. Google also says Gemma 4 improves OCR-style tasks such as chart reading and handwriting recognition; we found no independent accuracy figure, so test it on your own documents.

📍 In One Sentence

Camera questions run offline on Android through vision-capable models in apps such as Google AI Edge Gallery, LM Playground and Private Mind, and through Gemini Nano on supported phones.

💬 In Plain Terms

A vision model reads the picture itself; a text-only model never sees it, so the model you pick matters more than the app.

  • Google AI Edge Gallery: its Ask Image feature identifies objects, solves visual puzzles and describes images from the camera or gallery. Limit: needs a vision-capable Gemma model, and the README names no accuracy figures.
  • LM Playground: photo input for vision models in GGUF format. Limit: you must find and download a vision-capable GGUF plus its projector file; the app does not do that for you.
  • Private Mind: lists vision-capable models alongside text models. Limit: larger vision models can need more RAM than the 4 GB its listing recommends for bigger models.
  • LocalAI: Offline AI Chat LLM: supports vision-capable models according to its Google Play listing. Limit: closed source, and PromptQuorum's review rests on the store listing.
  • KernelAI: lists vision among its features. Limit: its claims rest on the developer's pages.
  • Ensu: accepts image input in chat. Limit: image support depends on the model you choose.

⚠️Warning: Do not use an on-device vision model for anything where a misread matters: medication labels, legal text, safety signs. Small vision models misread text and numbers, and there is no verified accuracy figure to lean on.

Voice: Dictation, Transcription, Read-Aloud, Assistants

Offline dictation and transcription are the most mature voice jobs on Android; an always-listening offline assistant is the least. The first two run a speech model such as Whisper on the phone and are triggered by you. The last needs a background service that Android is free to kill.

Google's ML Kit speech recognition has a basic mode that its documentation says is available on most Android devices with API level 31 or later, and an advanced mode on Pixel 10 phones with more devices in development. ML Kit also lists speech translation and text to speech; the overview page does not state device requirements for those, so check the feature pages.

For the full stack of offline voice, see Voice Assistants on Mobile, Offline, Local Whisper STT Compared, Local Voice and Audio Tools Compared and Build a Local Voice Assistant.

📍 In One Sentence

Offline dictation and recording transcription work on Android through Whisper-class models in apps like Private Mind, Ensu and LLM Hub, while offline always-on voice assistants remain limited by Android's background rules.

💬 In Plain Terms

Voice you start by tapping works offline; voice that must listen all day runs into battery limits and app killers.

Ensu

Voice job:
Local voice dictation into chat
Honest limit:
Not tested hands-on by PromptQuorum; no accuracy data.
Articles about Ensu (2)

Also mentioned in:

LM Playground

Voice job:
Voice dictation
Honest limit:
Dictation only; no spoken replies listed.
Articles about LM Playground (1)

LLM Hub

Voice job:
Whisper transcription alongside chat, image and translation
Honest limit:
Source-available under a noncommercial license; Premium tier is paid.
Articles about LLM Hub (2)

Also mentioned in:

MemoryAI

Voice job:
Hands-free voice conversation
Honest limit:
Closed source; check whether recognition is on-device in the privacy policy.
Articles about MemoryAI (4)

ToolNeuron

Voice job:
Voice input and output, plus an encrypted store
Honest limit:
No update for more than three months; see quiet apps.
Articles about ToolNeuron (4)

Also mentioned in:

TokForge

Voice job:
Text to speech with voice cloning
Honest limit:
Proprietary; voice cloning raises consent and impersonation questions.
Articles about TokForge (2)

RunAnywhere

Voice job:
SDK for speech-to-text, text-to-speech and voice activity detection in your own app
Honest limit:
A library for developers, not an app; commercial terms apply.
Articles about RunAnywhere (1)

Also mentioned in:

Home Assistant Assist

Voice job:
Voice control of your home with Whisper and Piper on your own hardware
Honest limit:
The speech models run on a home server, not on the phone.
Articles about Home Assistant Assist (2)

AquaVoice

Voice job:
Fast dictation that types into the active app
Honest limit:
Cloud service. It is here so you do not mistake it for local.
Articles about AquaVoice (2)

Also mentioned in:

⚠️Warning: Voice cloning and always-on recording are privacy and consent topics, not only features. Clone only your own voice or one you have written permission to use, and tell people when you record them.

Documents, PDFs and RAG

Document Q&A works on Android by splitting a file into chunks, finding the relevant ones and feeding only those to a small model. This is retrieval-augmented generation (RAG). It beats pasting a whole PDF into a small model, whose context window is too short for it.

The Gemini Nano Prompt API limits input to under about 4,000 tokens, which is roughly 3,000 English words, so system-level document questions need chunking too. For the method in depth, read Local RAG on Your PDFs, Step by Step and Best Local RAG Tools.

📍 In One Sentence

Android apps such as AnythingLLM Mobile, Private Mind, LM Playground and Honey LLM answer questions about your files on the phone by retrieving relevant chunks for a small model.

💬 In Plain Terms

The app searches your document for the right paragraphs and hands only those to the model, so a small model can answer questions about a long file.

  • AnythingLLM Mobile: MIT-licensed, Android only. Standalone mode runs a small model and a local vector database on the phone; connected mode pairs by QR code with AnythingLLM Desktop or Cloud for the full feature set. Limit: standalone quality is that of a small model; pairing is how you get more.
  • Private Mind: on-device document Q&A with source citations. Limit: citations show where an answer came from, not that it is right.
  • LM Playground: on-device document Q&A in a free MIT app. Limit: independent developer; check the review for supported file types before relying on it.
  • Honey LLM: answers questions about PDF, DOCX, TXT and Markdown files and keeps an encrypted local vault. Limit: young app on Google Play since March 2026; our review is listing-based.
  • LocalAI: Offline AI Chat LLM: document chat over PDF, Word, Excel and CSV using on-device vector retrieval. Limit: closed source; the review is listing-based.
  • ToolNeuron: document RAG with encrypted storage, voice, a local HTTP server and plugins. Limit: no update for more than three months.
  • MemoryAI: document Q&A plus an Agent Memory you can edit. Limit: closed source.
  • Caelum: document questions with optional web search. Limit: no update since August 2025, and the web search option sends queries online.

💡Tip: For a large archive, do not index it on the phone. Run the index on a computer or server and use the phone as the client, as the remote mode chapter describes.

Translation

Offline translation on Android works for common language pairs with small models, and quality drops for rare languages and long texts. Google says Gemma 4 natively supports over 140 languages and lists speech translation among the ML Kit GenAI APIs; neither claim comes with an accuracy figure we could verify, so test your own language pair before relying on it.

📍 In One Sentence

Android apps like LLM Hub and Google AI Edge Gallery translate text and recordings on the phone, and ML Kit adds speech translation on supported devices.

💬 In Plain Terms

Small models translate everyday sentences well and struggle with rare languages, idioms and legal wording.

  • LLM Hub: on-device translation next to chat, an agent with MCP servers, image and music generation and Whisper transcription. Limit: source-available under a noncommercial license, and a paid Premium tier.
  • Google AI Edge Gallery: its Audio Scribe feature transcribes and translates voice recordings on-device. Limit: experimental beta.
  • Local AI Mail: an Android email client that summarizes, translates and drafts replies with an on-device model and can switch to Gemini with your own key. Limit: a very early release; the optional Gemini mode sends mail text to Google.

⚠️Warning: Do not rely on a small local model for contracts, medical texts or anything where a mistranslation has consequences. Use it for gist and drafts, then verify.

Image Generation and Editing

Image generation on a phone is possible but slow compared with a desktop GPU, and it is the job where chip-specific tuning matters most. Samsung and Nota AI describe EdgeFusion, a Stable Diffusion variant that generates 512 by 512 images in under a second on the Exynos 2600 NPU (Samsung and partner claim, quoted in our Galaxy S26 guide). That is a vendor-tuned pipeline; we found no verified speed for the sideloaded apps below, so we give none.

For the cloud-versus-local trade-off in general, see Local AI Image Generation vs Cloud. Editing tools such as inpainting are rarer on phones than text-to-image.

📍 In One Sentence

Android apps such as TokForge, Local AI by GeetMark, Laria and LLM Hub generate images with on-device models, but no source we found publishes verified generation times for them.

💬 In Plain Terms

Your phone can draw a picture without the internet, but expect to wait much longer than on a computer with a graphics card.

  • Local AI: Offline Chat and Image: free, with chat on 20+ models, Stable Diffusion images and 15 prompt toolkits. Limit: contains ads, source code and license not published.
  • TokForge: roleplay characters with on-device image generation, voice and document Q&A. Limit: proprietary; speed results can be posted to a public leaderboard, so check what the benchmark shares.
  • Laria: image tools alongside local and cloud chat. Limit: paid; some image tools may use cloud providers, so check each tool.
  • LLM Hub: image and music generation in the same app as chat and translation. Limit: noncommercial source-available license.

💡Tip: Heat is the practical limit. A few images in a row can warm the phone enough to slow it. Generate in short batches and let it cool.

Notes, Knowledge and Bookmarks

Notes and bookmark apps give you a mobile client for knowledge that lives on your server or in plain files, and the AI features run wherever you point them. None of the apps below runs a language model inside the Android app itself. They connect to an AI provider you choose, which can be a local Ollama server on your network or a cloud service. That makes them local-first for storage, not necessarily for inference.

📍 In One Sentence

Android clients for Karakeep, Linkwarden, Blinko, Logseq, SiYuan and Hillnote keep notes and bookmarks under your control, while their AI features run on a server or provider you configure.

💬 In Plain Terms

These apps store your knowledge locally; the AI model that tags or summarizes it can be local too, but only if you set it up that way.

  • Karakeep: AGPL-3.0, self-hosted with Docker, saves links, notes, images and PDFs and uses AI to tag, summarize and index them. It has Android and iOS apps and browser extensions. Limit: AI tagging needs a model endpoint you configure.
  • Linkwarden: AGPL-3.0 bookmark manager that saves a copy of each page and can tag links with a local Ollama model or a cloud provider. Limit: Cloud edition is paid; self-hosting is on you.
  • Blinko: GPL-3.0, self-hosted note-taking with RAG-powered natural-language search over your own notes, run as a Docker container. Limit: the RAG search uses an AI provider you connect.
  • Logseq: AGPL-3.0, local-first outliner with Android and iOS apps. It has no built-in model; the database-version beta adds an optional MCP server so an external AI client can read and edit a graph. Limit: AI is bring-your-own and the MCP server is in beta.
  • SiYuan: AGPL-3.0 block-level knowledge workspace with Android and iOS apps and built-in AI agent collaboration. Limit: encrypted cloud sync is a paid add-on; check where its AI features run before assuming they are local.
  • Hillnote: Markdown notes workspace for Mac, Windows, iPhone, iPad and Android that stores files on your disk. Its on-device AI is listed as Ollama and Apple Intelligence, neither of which runs on an Android phone, so check which AI features the Android app actually exposes before you buy.

💡Tip: A good pattern: self-host the notes app and a small Ollama model on a home server, and let the Android app be the capture and search client.

Smart Home and Agents

For smart homes the phone is a remote control and a microphone; for agents it is both a place to run one and, increasingly, the thing an agent controls. Smart-home AI belongs on a home server, because a phone sleeps, moves and gets killed. Agents on Android split into three families: assistants that use tools, agents that live on a server and use the phone as a client, and agents that operate the phone itself.

For the smart-home side in depth, see the PromptQuorum smart-home cluster; this chapter covers what the phone does.

📍 In One Sentence

Android phones act as clients for smart-home AI (Home Assistant Assist, Scrypted, EverShelf, LocalGhost) and can run tool-using agents (Kai 9000, PicoClaw, Aori), but system-level control of other apps is still in preview.

💬 In Plain Terms

Let a home server be the brain, and let the phone be the remote and the microphone.

Phone as a smart-home client

  • Home Assistant Assist: the voice assistant inside Home Assistant. It can run fully on your hardware with Whisper or Speech-to-Phrase for recognition and Piper for speech over the Wyoming protocol, and can hand requests to an LLM such as one served by Ollama. The Android app is the microphone and screen. Limit: the speech and language models run on your home server, not on the phone.
  • Scrypted: open-source camera and NVR platform with Android and iOS viewer apps, plus optional local LLM summaries through llama.cpp on the server. Limit: the NVR component is paid, and the AI runs where Scrypted runs.
  • EverShelf: self-hosted pantry manager with barcode scanning, expiry tracking and shopping lists; AI features are optional and use Gemini, OpenAI or your own Llama server. Limit: it is not a chat or LLM app, and the optional cloud providers see what you send.
  • LocalGhost: MIT-licensed personal AI system on a Debian machine you own, with an Android app as the only client, reading photos, notes, voice notes and health data with local models. Limit: very early (the reviewed release was v0.0.7), and health data is sensitive, so treat it as an experiment.

Agents that use tools

  • Kai 9000: Apache-2.0 assistant with persistent memory and tools on Android, iOS, desktop and web. It talks to cloud providers by default and can optionally run a downloaded model, including Gemma 4, on the device. Limit: cloud is the default, so switch to on-device deliberately.
  • PicoClaw: MIT-licensed agent from Sipeed, a single Go program for small boards, Android phones and desktops that forwards requests to a model you configure (a cloud provider, or your Ollama or vLLM server). Limit: the model is not on the phone.
  • Aori: free beta personal agent with persistent memory, personas and scheduled routines for desktop and Android. Limit: the Android app sends prompts to the provider you connect, and it has been quiet for over three months (see quiet apps).
  • Atomic Bot: one-click packaging of the open-source OpenClaw agent stack, local or on the maker's paid cloud. Limit: the directory lists mobile, but our review found no documented Android build on the maker's site, so verify before you plan around it.

Agents that drive the phone

  • Android AppFunctions lets apps expose functions that an agent such as Gemini can call. Google documents it as a platform API with a Jetpack library for Android 16 and later; sources we read disagree on whether it is in limited or private preview, so treat it as not generally available. It is the on-device counterpart of MCP tools, and it needs the app's developer to opt in.
  • ML Kit AgentRunner gives developers a multi-step tool-calling loop on top of the Gemini Nano Prompt API, so an app can run a small agent entirely on the phone. It carries the same limits as the Prompt API: foreground only, supported phones only.
  • Open-AutoGLM is an open-source phone-agent framework from the Zhipu team (zai-org/Open-AutoGLM) that runs on a computer and operates an Android phone over ADB, with a vision-language model reading the screen. The model can be a remote API or a locally served model, and the agent itself is not on the phone. Check its license and the model card yourself; our sources disagreed.

⚠️Warning: An agent that can tap, type and read your screen can also leak what it sees. Run phone-driving agents on a test phone or a work profile, never on the phone that holds your banking apps.

Developer Paths

Android has many routes to on-device AI, and picking one comes down to whether you need system integration, model choice or the NPU. The table compares the routes; the notes after it give the caveats that cost people the most time.

📍 In One Sentence

Developers can target Android with Google ML Kit GenAI and Gemini Nano, LiteRT and LiteRT-LM, llama.cpp through Termux or the NDK, MLC LLM, ExecuTorch, Qualcomm GenieX, or SDKs such as RunAnywhere and Llamatik.

💬 In Plain Terms

Use Google's system model when it is enough, a Google or Qualcomm runtime when you need the NPU, and llama.cpp when you want any GGUF model.

Reviewed developer tools

  • RunAnywhere: SDK for LLM, speech-to-text, text-to-speech, voice activity detection, vision-language models and embeddings that run on the device, for iOS, Android, Flutter and React Native apps. Limit: Apache-2.0-based license with commercial terms.
  • Llamatik: the Kotlin Multiplatform SDK behind the offline chatbot of the same name. Limit: one independent developer.
  • Foundry Local: Microsoft's on-device runtime with SDKs for C#, JavaScript, Rust and Python. Limit: the review documents the desktop SDKs and no Android build.
  • MLC LLM: the compiler and runtime behind MLC Chat, targeting iPhone, Android, browser (WebGPU) and desktop GPUs. Limit: a compile step per model and target.
  • OlliteRT: turns an Android phone into an OpenAI-compatible server on Google's LiteRT-LM, with no chat interface of its own. Limit: it needs .litertlm model files, not GGUF.
  • Mobile LM Server: runs GGUF and LiteRT-LM models and serves them through OpenAI- and Ollama-compatible endpoints. Limit: contains ads and in-app purchases; our review is listing-based and, per that review of the listing, noted that its privacy policy and Data safety card differ on location.
  • Ollama Local AI: runs GGUF models through an embedded llama.cpp engine and exposes an OpenAI-compatible API on your Wi-Fi so tools such as Cursor can use the phone. Limit: it is not the Ollama project's app, and it is free with in-app purchases.

ML Kit GenAI and Gemini Nano (AICore)

Best for:
Summaries, rewriting, image description and a Prompt API on a system model that is updated for you. Includes AgentRunner and Embedding APIs. Prompt API input stays under about 4,000 tokens.
Main limit:
Foreground use only; quotas per app; supported phones only; not supported with an unlocked bootloader (Prompt API).

AICore Developer Preview with Gemma 4

Best for:
Trying Gemma 4 E2B and E4B through the ML Kit Prompt API; code written now is meant to carry over to Gemini Nano 4 devices.
Main limit:
A preview; Google said tool calling, structured output, system prompts and thinking mode were still being added.

LiteRT and LiteRT-LM

Best for:
Your own app with Gemma, Llama, Phi-4 or Qwen families, GPU and NPU acceleration, vision and audio input, function calling.
Main limit:
NPU needs per-chip compiled models; Swift and JavaScript are early previews.

Qualcomm AI Hub and GenieX

Best for:
Snapdragon 8 Elite and 8 Elite Gen 5, GGUF through llama.cpp with a Hexagon backend or pre-compiled AI Hub bundles through AI Engine Direct.
Main limit:
Snapdragon only; the Android runtime rejects non-default GPU layer and context settings on the AI Hub path.

MediaTek NeuroPilot through LiteRT

Best for:
Dimensity NPUs with pre-compiled Gemma and Qwen3 models.
Main limit:
Custom Gemma conversion for MediaTek was not yet available.

Google Tensor SDK

Best for:
Pixel 10 TPU through LiteRT, with a Model Garden of 100+ models.
Main limit:
Beta; Pixel 10 family only.

llama.cpp in Termux or the NDK

Best for:
Any GGUF model on the CPU, no root; a one-off prototype or a CLI.
Main limit:
Android process limits and heat; start with a context near 4096 or the terminal can be killed.

MLC LLM

Best for:
Compiling a model for one target with GPU acceleration.
Main limit:
Compile toolchain; OpenCL support varies by phone.
Articles about MLC LLM (7)

Also mentioned in:

ExecuTorch

Best for:
PyTorch export to Android, listed as covering LLMs and CPU, GPU, NPU and DSP backends.
Main limit:
We could not confirm the exact backend list or status from the overview page.

SDKs: RunAnywhere, Llamatik, Foundry Local

Best for:
Speech, vision and LLM features in your own app (RunAnywhere); Kotlin Multiplatform (Llamatik); Microsoft runtime with C#, JavaScript, Rust and Python SDKs (Foundry Local).
Main limit:
Foundry Local is listed for Android in the directory, but its review documents no Android build; confirm before use.

💡Tip: Read the maker pages: ML Kit GenAI overview, Gemini Nano and AICore, LiteRT-LM, Qualcomm AI Hub, Termux and the llama.cpp Android guide.

Remote Mode: Phone as a Thin Client

The most capable "local AI on Android" setup is often a phone that talks to a model on your own computer. The model runs on your Ollama or LM Studio machine; the phone is the interface. You keep privacy, because traffic stays on your network (or a VPN), and you get 7B to 70B class models without touching your phone's RAM.

Three cautions. First, the server must be on and reachable, so the setup fails away from home unless you add a VPN such as WireGuard or Tailscale. Second, Ollama listens on the local machine only by default, and exposing it to your network, for example by setting its host address, gives anyone on that network access because it has no built-in login; use a reverse proxy with authentication or a VPN. Third, a cloud API key inside the same app is a different data path, so keep local and cloud profiles clearly separate.

📍 In One Sentence

Android apps such as LocalMind, PocketLLM Lite, AnythingLLM Mobile, AnyLLM and RikkaHub let the phone act as a client to your own Ollama or LM Studio server on your network.

💬 In Plain Terms

The phone is the screen and keyboard; the heavy thinking happens on your computer at home.

  • LocalMind: free, MIT-licensed chat app for Android and iPhone that sends prompts to a model on the phone or to Ollama, LM Studio, OpenRouter or any OpenAI-compatible endpoint, with MCP tools. Limit: independent developer; the remote path sends text to whichever endpoint you pick.
  • PocketLLM Lite: MIT-licensed Android app that runs Cactus-format models on the phone or connects to your own Ollama or OpenAI-compatible server. Limit: no named developer, and PromptQuorum has not installed it.
  • AnythingLLM Mobile: connected mode pairs with AnythingLLM Desktop or Cloud over the local network by QR code. Limit: Android only.
  • AnyLLM: Android client for your own API keys (OpenAI, Anthropic, Google, Groq) that can also connect an Ollama server, with MCP tools and AI-versus-AI debates. Limit: closed source, and cloud keys mean prompts leave your network.
  • RikkaHub: open-source Android client for cloud providers that can reach a self-hosted Ollama or LM Studio server through its custom-endpoint field. Limit: it does not run models on the phone, so it is not an offline app by itself.
  • Chapper: a polished Ollama and LM Studio client, but Apple only (iPhone, iPad, Mac). Limit: no Android version exists, so Android users should use one of the apps above.

💡Tip: A phone can also be the server. OlliteRT, Mobile LM Server and Ollama Local AI serve their model to other devices on your Wi-Fi, which is useful for testing and quick scripts but not a replacement for a real server.

Privacy and Telemetry Reality

On-device inference keeps your prompts off a vendor's inference servers; it does not by itself stop the app from sending data elsewhere. "Zero telemetry" can mean no analytics SDK, no crash reports, no ads and no network calls after model download, or only some of those. Apps vary, and the claim is the maker's unless you verify it.

Three places to check. The Google Play Data safety card lists what the developer says it collects or shares; it is developer-declared. The privacy policy says what the company does with data; the Mobile LM Server review, which is based on the listing, noted the policy and the card differing on location. The network behavior itself: a firewall or network monitor that shows connections after the model is downloaded is the most direct test.

Google's AICore design is a useful reference point: Google documents that AICore has no direct internet access, that model downloads go through the open-source Private Compute Services companion, and that it does not store the input or output after processing. That is Google's description of its own system, not an independent audit. Apps built on other engines make their own promises: ToolNeuron, for example, states "no telemetry" as a design goal in its README, while the Google AI Edge Gallery review of the iPhone build found diagnostics in the privacy label.

📍 In One Sentence

Offline inference keeps prompts on the phone, but "zero telemetry" must be checked in the Data safety card, the privacy policy and the app's actual network traffic.

💬 In Plain Terms

Running the model on your phone protects what you type; it does not promise the app never talks to the internet.

Data safety card

Where:
Google Play listing
What it can reveal:
Developer-declared data collection and sharing, which may be incomplete or outdated.

Privacy policy

Where:
Developer website
What it can reveal:
Retention, third parties, jurisdiction, and whether the policy matches the card.

Permissions

Where:
Android settings, App info
What it can reveal:
Microphone, camera, storage or network access that a purely local app should not need.

Optional cloud features

Where:
App settings
What it can reveal:
Web search, cloud providers and sync toggles that quietly widen the data path.

Network traffic

Where:
Airplane mode test or a firewall app
What it can reveal:
Whether chat works with all network access blocked after download.

Source code

Where:
GitHub or F-Droid
What it can reveal:
Whether claims can be audited at all; closed-source claims rest on trust.

💡Tip: The airplane-mode test is simple and honest: download the model on Wi-Fi, enable airplane mode, then chat. If the app refuses to work, it needs the network. If it works, it still may have sent data earlier, so combine it with the permission and policy checks.

⚠️Warning: Obscurify is on Android and markets privacy, but it is a cloud service: it reaches 200+ cloud models behind a random account number, with an optional local mode that runs GGUF models in the browser. Anonymous cloud is not local AI, and AquaVoice is cloud dictation. Know which side of the line each tool is on.

Battery, Heat and Background Kills

Three things limit local AI on a phone before the model does: heat, battery and Android's habit of killing background work. All three get worse with larger models and longer sessions, and none of them shows up in a spec sheet.

📍 In One Sentence

On Android, sustained local inference heats the phone and drains the battery, and the system or the brand's battery manager may kill the app mid-answer unless you exempt it.

💬 In Plain Terms

Running a model is like running a game: the phone warms up, slows down to protect itself, and may close the app if it thinks it is wasting power.

  • Heat. A phone cannot hold peak speed for long. It slows itself to stay cool, so the first answer is usually faster than the tenth. We have no sourced throttling figure and give none; test with a ten-minute run on your own phone. Remove the case, avoid the charger while generating, and prefer smaller models for long sessions.
  • Battery. Generation keeps the CPU, GPU or NPU busy, so expect noticeable drain. Google says Gemma 4 on AICore uses up to 60% less battery than previous versions and runs up to 4 times faster (Google's claim for its own models and phones). Those numbers do not transfer to a sideloaded GGUF app.
  • Foreground rules for Google's system model. Google documents that ML Kit GenAI inference is allowed only when the calling app is the top foreground app, with errors for background use and per-app quotas for rapid or long-running use. Apps that use Gemini Nano cannot summarize in the background.
  • Process limits. Android 12 introduced a limit on background child processes, which is why Termux can print "signal 9" and stop. The Termux maintainers warn that it may be unstable on Android 12 and later. On recent versions, guides describe a Developer options switch named "Disable child process restrictions" and, on Android 12 to 13, an ADB setting; details differ by brand and version, so verify on your phone before relying on either.
  • Brand battery managers. Samsung, Xiaomi, OnePlus and others add their own sleeping-app lists. Exempt your AI app from battery optimization, lock it in the recent apps view, and keep the screen on during long answers.
  • Storage. Models are large files. Store them in the location the app expects, keep several gigabytes free, and delete models you no longer use.
  • Context size. A larger context window uses more memory. The llama.cpp Android guide suggests starting near 4096 because larger values can spike memory and kill the terminal.

⚠️Warning: Do not leave a phone generating images or long answers under a pillow or in a hot car. Heat shortens battery life, and a throttled phone is a slow phone.

Not Possible, or Not Yet

These are the limits that no app can fix today, grouped by why. Knowing them saves hours of trying settings that will not help.

📍 In One Sentence

Local AI on Android cannot match cloud models on hard tasks, cannot hold 70 billion parameter models or very long contexts, and cannot run as an always-on assistant that Android will not kill.

💬 In Plain Terms

A phone has a small memory, a small battery and a hot, sealed case; those three facts set the ceiling.

Frontier-model quality offline

Why:
Phone models are 1 to 8 billion parameters; cloud flagships are far larger. Small models are weaker at hard reasoning and long, exact answers.
What to do instead:
Use local for private, simple or offline jobs; use a server or cloud for hard ones.

70 billion parameter models

Why:
At 4-bit, the weights alone need tens of gigabytes; most phones have 8 to 16 GB in total.
What to do instead:
Run them on a computer and use remote mode.

Very long documents in one go

Why:
Context windows are small and memory grows with context; the Gemini Nano Prompt API stays under about 4,000 tokens.
What to do instead:
Chunk with RAG; index on a server.

Fast replies for long outputs

Why:
Speed is bounded by memory bandwidth, and heat throttles sustained runs.
What to do instead:
Pick a smaller model; ask for short answers.

A reliable always-on background assistant

Why:
Android restricts background work, and brand battery managers kill apps; Google's system model is foreground-only.
What to do instead:
Put the assistant on a home server; use the phone as a client.

NPU speed from any sideloaded model

Why:
NPU paths need models compiled per chip; only some operations are supported, and the rest fall back to CPU or GPU.
What to do instead:
Use LiteRT, GenieX or NeuroPilot-ready models, or accept CPU and GPU speed.

Gemini Nano in any app on any phone

Why:
Access depends on AICore, device and chipset support, and a locked bootloader for the Prompt API.
What to do instead:
Use an engine you control, such as llama.cpp or LiteRT-LM.

Training or fine-tuning on the phone

Why:
Memory, compute and thermal budgets are too small for anything beyond tiny experiments.
What to do instead:
Fine-tune on a computer, then load the result.

Verified accuracy for small vision and OCR models

Why:
We found no independent accuracy figures for the models these apps run.
What to do instead:
Test on your own samples; never trust blindly.

Guaranteed zero telemetry

Why:
An app can run locally and still collect analytics; the claim is the maker's.
What to do instead:
Check the Data safety card, the policy, permissions and traffic.

One setup that works on every Android phone

Why:
Chips, RAM, OS versions and battery managers differ by brand and model.
What to do instead:
Test one small model first, then scale up.

💡Tip: Several of these limits are moving. Gemma 4 on Gemini Nano 4, wider LiteRT NPU support and AppFunctions are all in preview or beta in the sources we read. Re-check them at the next refresh rather than assuming today's limits are permanent.

Quiet Apps to Check Before You Rely on Them

Some apps in the directory have had no upstream update for more than three months. This is not a statement that the project is dead or unsafe; it is a prompt to check the maker's page, the store listing and the issue tracker before you build a routine around the app. The dates come from the directory entries, which record the latest release or last substantive commit read from a primary source.

📍 In One Sentence

Several Android-listed apps in the directory, including MLC Chat, Caelum, ToolNeuron, PAIOS, Aori and Atomic Bot, had no upstream update for more than three months at the time of writing.

💬 In Plain Terms

A quiet app may still work, but nobody may fix a bug or a security problem soon.

  • MLC Chat: the directory's last recorded update is from September 2024; check the maker's GitHub releases for newer builds. The MLC LLM runtime itself keeps moving (MLC LLM), so for current work build from the runtime instead.
  • Caelum: no update since August 2025, more than a year.
  • ToolNeuron: a quiet period of several months; a good fit (MIT license, encrypted storage), but check recent activity.
  • PAIOS: the reviewed release is from spring 2026 and the project labels itself alpha; check that your phone is still supported by AICore.
  • Aori: a free beta without a recent release; the Android app sends prompts to a cloud provider you connect.
  • Atomic Bot: no recent release, and no documented Android build in our review.

Decision Path: I Have This Phone and Want This

Start from your RAM and your goal, not from an app's popularity. Find the row that matches your phone, then the goal, then follow the link.

📍 In One Sentence

Match your phone's RAM and chip to a model size, then match your goal to a chapter: chat on PocketPal AI, camera and Gemma on Google AI Edge Gallery, documents on AnythingLLM Mobile, and heavy models on a server through a remote client.

💬 In Plain Terms

Small phone, small model; big job, big server.

  • 4 to 6 GB, want simple private chat: PocketPal AI with a model of 1 to 2 billion parameters, or Ensu if you have at least 4 GB.
  • 8 GB, want the best everyday balance: PocketPal AI or Google AI Edge Gallery with a 3 to 4 billion parameter model.
  • 12 to 16 GB, want the biggest model that fits: a 7 to 8 billion parameter model at low-bit quantization in a GGUF runner such as Maid, with a short context.
  • Pixel 9 or Pixel 10: try Google's system model through PAIOS, and use Google AI Edge Gallery for Gemma models.
  • Galaxy S25 or S26: read Galaxy S26 On-Device AI first, because Galaxy AI is hybrid and the chip depends on model and region.
  • Snapdragon 8 Elite flagship, want maximum speed: developers can use LiteRT and GenieX; as a user, start with Google AI Edge Gallery and compare it with a llama.cpp app.
  • Want to ask questions about my files: AnythingLLM Mobile or Private Mind on the phone; a home server for big archives.
  • Want translation or dictation offline: LLM Hub, Private Mind or Ensu, after testing your language.
  • Want to draw images: Local AI by GeetMark or TokForge, with patience and a cool phone.
  • Want my home to use it: Home Assistant Assist with a home server, and the phone as the remote.
  • Own a computer with a GPU: run Ollama or LM Studio there and use LocalMind or AnythingLLM Mobile as the phone client.
  • Developer: pick from the developer paths table; start with the ML Kit Prompt API if you only need small text tasks, and LiteRT-LM if you need your own model.

First Run in 15 Minutes

You can have a working offline assistant in about 15 minutes, and most of that time is the model download. This walkthrough uses PocketPal AI as the example; the steps are the same in any GGUF runner. Use Wi-Fi and a charger for the download.

📍 In One Sentence

Install one app, download a model sized to your RAM on Wi-Fi, run a prompt, then test again in airplane mode to confirm the answer works offline.

💬 In Plain Terms

Download once with internet, then switch the internet off and see if it still answers.

  1. 1
    Check storage and RAM (1 minute)
    Why it matters: Models are gigabytes; free storage of 3 to 6 GB covers a first model, and RAM decides the size. Look under Settings, About phone.
  2. 2
    Install one app (2 minutes)
    Why it matters: Use [PocketPal AI](/power-local-llm/pocketpal-ai-review) or [Google AI Edge Gallery](/power-local-llm/google-ai-edge-gallery-review) from Google Play. One app at a time keeps results comparable.
  3. 3
    Join Wi-Fi and plug in (1 minute)
    Why it matters: The download is large and heating while charging is acceptable; mobile data may be slow and costly.
  4. 4
    Choose a model by RAM (1 minute)
    Why it matters: 4 to 6 GB: 1 to 2B. 8 GB: 3 to 4B. 12 to 16 GB: 4 to 8B. When in doubt, pick the smaller one.
  5. 5
    Download and load the model (5 to 7 minutes)
    Why it matters: The time depends on your connection and the file size; keep the screen on until it finishes.
  6. 6
    Send a first prompt (1 minute)
    Why it matters: Ask something simple. Note how fast the words appear; this is your baseline.
  7. 7
    Test in airplane mode (1 minute)
    Why it matters: Turn on airplane mode and ask again. If it answers, inference is local for this model and app.
  8. 8
    Exempt the app from battery optimization (1 minute)
    Why it matters: This reduces mid-answer kills on brands with aggressive battery managers.
  9. 9
    Try one job from this guide (as long as you like)
    Why it matters: Dictation, a document question or an image question tells you quickly whether the phone is a good fit.

💡Tip: If the first model feels slow or crashes, go one size down before changing apps. If it feels weak, go one size up or use remote mode.

Summary: Phone Tier, App, Model Size

One table to screenshot. These are starting points based on the RAM arithmetic above and on what each app documents, not benchmark results.

📍 In One Sentence

Use 1 to 2 billion parameter models on 4 to 6 GB phones, 3 to 4 billion on 8 GB phones, 7 to 8 billion on 12 to 16 GB phones, and a server for anything larger.

💬 In Plain Terms

Match the model to your RAM first, then choose the app for the job.

Phone tier
Good first app
Model size
Best for
4 to 6 GB, budgetPocketPal AI1 to 2BShort chat, simple rewriting
8 GB, mid-rangeGoogle AI Edge Gallery3 to 4BEveryday chat, camera questions
12 GB, flagshipMaid4 to 8BBetter answers, short context
16 GB, gaming phoneLM Playground7 to 8BLarger local models
Pixel 9 or 10PAIOSGemini NanoSystem model, no download
Any phone + home serverLocalMind7 to 70B on serverHeavy models, privacy at home
Documents on any tierAnythingLLM MobileSmall model + RAGQuestions about your files

Common Mistakes

Most failed first attempts come from the same handful of mistakes.

  • Choosing a model too large for free RAM. The app crashes or the phone freezes. Go one size down.
  • Assuming the NPU is used. Most sideloaded apps run on the CPU or GPU, so a "50 TOPS" chip does not make them faster.
  • Treating "offline" as "private". Offline inference is one layer; analytics, ads and cloud toggles are another. Check the Data safety card and policy.
  • Installing Termux from several sources. Mixing Google Play, F-Droid and GitHub builds breaks signatures and updates. Pick one and stay on it.
  • Using a text-only model for vision. Images need a vision-capable model and, in some apps, an extra projector file.
  • Expecting a background assistant. Android and brand battery managers kill idle apps; put always-on jobs on a server.
  • Judging a model by the first answer. Cold phones are fastest; run a ten-minute test before you decide.
  • Installing an Apple-only app. Chapper has no Android version. Use an Android client instead.
  • Trusting small models with high-stakes text. Translation, OCR and summaries can be wrong; verify before acting.

Sources

Platform and chip facts come from the maker pages below, read in November 2026. Product facts come from the linked PromptQuorum reviews and the directory entries, which cite their own primary sources.

Frequently Asked Questions

Can I run local AI on Android without internet?

Yes. Download an on-device app such as PocketPal AI or Google AI Edge Gallery and a 1 to 4 billion parameter model on Wi-Fi, then use it in airplane mode. The first download needs a connection; chat, dictation, translation and document questions can work offline afterwards, with quality below cloud AI.

How much RAM do I need to run an LLM on my Android phone?

As a rule of thumb, 4 to 6 GB fits 1 to 2 billion parameter models, 8 GB fits 3 to 4 billion, and 12 to 16 GB can hold 7 to 8 billion at low-bit quantization. Android and other apps use several gigabytes, so free RAM matters more than the number on the box. Apps may set their own floors.

Do Android apps use the NPU for local LLMs?

Usually not. Generic GGUF apps run on the CPU and sometimes the GPU. NPU paths exist through Google LiteRT, Qualcomm GenieX and MediaTek NeuroPilot, but they need models prepared for a specific chip. Google reports large speed-ups for its own examples, such as Gemma 3n E2B on a Dimensity 9500 NPU, but those are developer paths.

What is Gemini Nano and can any app use it?

Gemini Nano is Google's on-device model, managed by Android's AICore service. Apps reach it through ML Kit GenAI on supported phones, such as Pixel 9 and 10 and recent Galaxy and Xiaomi models. Google says it runs only while the app is in the foreground, and the Prompt API is not supported with an unlocked bootloader.

What is the best local AI app for Android?

There is no single best. PocketPal AI is a free, open-source GGUF runner with a wide model choice; Google AI Edge Gallery is Google's own app for Gemma-family models; AnythingLLM Mobile fits document questions; LocalMind fits a phone that talks to your own server. Our Android roundup compares the best-known options.

Is local AI on Android really private?

When an app runs the model on the phone, prompts are not sent to an AI vendor's inference servers for that step. The app can still collect analytics or use cloud features. Check the Data safety card, the privacy policy, permissions and whether the app works with airplane mode on after the model download.

Can I run Ollama on an Android phone?

There is no official Ollama app for Android. A community route runs it in Termux on the CPU, and you can run llama.cpp in Termux directly, or use an Android app that mimics an Ollama-compatible API, such as Mobile LM Server or Ollama Local AI (not made by the Ollama project). Most people get better results by running Ollama on a computer and using a phone client.

Does Android allow local AI in the background?

Only partly. Android restricts background processes, brand battery managers add their own limits, and Google's Gemini Nano APIs work only in the foreground. Exempt the app from battery optimization and keep the screen on for long answers; put always-on assistants on a home server.

Can a phone run a 70 billion parameter model?

Not realistically. At 4-bit, the weights alone need tens of gigabytes, while phones typically have 8 to 16 GB in total. Run such models on a computer and use your phone as a client through an app such as LocalMind or AnythingLLM Mobile.

How do I translate text offline on Android with local AI?

Use an app that includes translation, such as LLM Hub, or Google AI Edge Gallery's Audio Scribe for recordings. Small models handle common language pairs reasonably and struggle with rare languages and legal wording, so test your pair and verify important text.

Is it safe to use an on-device vision model for documents or labels?

Not for high-stakes reading. Small vision models can misread text and numbers, and we found no independent accuracy figures for the models these apps run. Use them for gist, then verify against the original.

What is different on an iPhone?

Both are capable, but they trade differently. iPhone has one vendor and fewer device variants, so results tend to be more predictable. Android gives you more room to install apps from outside the store and to use Termux, which is an Android app. Apple's rules for installing apps from outside the App Store differ by region and change over time, so check Apple's current policy for your country. Our sibling guide, Local AI on iPhone, covers the Apple side, and Galaxy vs iPhone On-Device AI compares the two ecosystems.

← Back to Power Local LLM

MCP