On-device AI is artificial intelligence that runs its calculations on your phone's own chip instead of sending your request to a company's server. Apple, Samsung, and Google have each built a version of it into their newest phones in 2026, and all three describe it the same way in their own documentation: faster, more private, and available without an internet connection. This explainer works from each company's official technical material to show what that actually means, where the three approaches genuinely differ, and where "on-device" quietly stops being true.
This explainer is based on official developer documentation, security research publications, and hardware product briefs verified as of 13 September 2026.
What changes when a model runs on the phone instead of the cloud
A cloud AI model like the one behind ChatGPT or a full-size Gemini response lives on a data center server. Your phone sends it text, the server does the thinking, and the answer comes back over the network. On-device AI moves that thinking onto a small, compressed model that fits inside the phone's own storage and runs on a dedicated chip block called an NPU — a neural processing unit, built specifically for the kind of math AI models use.
Apple's own machine learning research team describes this trade-off plainly: on-device models are deliberately smaller so they can run within a phone's power and memory limits, while cloud models stay large because a data center has neither constraint. Google's Android developer documentation for Gemini Nano says the same thing from the other direction — on-device generative AI models "are significantly smaller and less generalized than their cloud-based equivalents" specifically because they run on devices with less computational power than servers.
How a single request actually gets routed, step by step
Apple's Foundation Models framework is the clearest public illustration of this because Apple documents the decision path directly for developers building apps.
- The request first goes to the on-device model. Apple's on-device foundation model, part of the third-generation Apple Foundation Models introduced in 2026, handles the request locally if it's simple enough — summarizing a note, drafting a reply, rewriting a paragraph.
- The on-device model has a hard capacity limit. Apple's own technical documentation states it processes a maximum of 4,096 tokens per session — roughly 3,000 words of combined input and output, in a single conversation.
- If the task needs more context or heavier reasoning, it escalates to Private Cloud Compute (PCC). PCC is Apple's server infrastructure, but Apple's security research team designed it so that, in their words, personal data sent to PCC "isn't accessible to anyone other than the user — not even to Apple." The cloud model that runs there supports a 32,000-token context — eight times larger — and adds reasoning and image-input capability the on-device model doesn't have.
- The two paths have different limits by design, not by accident. Apple's own WWDC26 developer session states it explicitly: the on-device model works offline with no request limits, while PCC needs a connection and has a daily usage limit.
Samsung's Galaxy AI and Google's Gemini Nano follow the same basic shape — small local model for simple, privacy-sensitive tasks, larger cloud model for anything heavier — but they hand the decision to the user rather than making it automatically for every request.
Three companies, three different defaults
The underlying architecture — small local model, larger cloud model, an escalation path between them — is close to identical across Apple, Samsung, and Google. What differs, and what actually matters for a buyer, is who controls the escalation and what happens by default.
| Platform | Local model | Cloud fallback | Default behavior | User control |
|---|---|---|---|---|
| Apple Intelligence | ~3B-parameter on-device model, 4,096-token limit | Private Cloud Compute, 32K-token context, daily limit applies | Automatic escalation decided by the OS per request | No manual toggle for individual requests; PCC is architected so Apple itself cannot access the data |
| Samsung Galaxy AI | Runs features like Audio Eraser and basic Note Assist fully offline | Generative Edit and other heavier features require Samsung's servers | Hybrid — some features default to cloud processing even when a local option exists | An explicit "Process data only on device" toggle in Settings forces local-only processing across features |
| Google Gemini Nano | Runs inside Android's AICore system service on supported Pixel and partner devices | Full Gemini models in the cloud for anything Nano can't handle | App-by-app — each app decides whether to call Nano or a cloud API | No phone-wide toggle; control lives with whichever app you're using |
Apple's model is the most restrictive but also the most automatic — you can't manually force everything to stay local, but Apple's architecture is built so the cloud tier can't retain or expose your data either. Samsung's is the most transparent about the trade-off, because it puts a single visible switch in front of the user and states outright that some features send data to Samsung's servers unless that switch is on. Google's is the least visible of the three, since the local-versus-cloud decision happens inside whichever third-party app you're using, with no system-wide setting to check.
What "on-device" still doesn't cover
Each company's own documentation is honest about the limits, once you read past the marketing language.
- Apple: anything needing more than 4,096 tokens of context, image generation beyond Apple's own on-device diffusion model, or complex multi-step reasoning goes to PCC — meaning it needs a network connection and is subject to a daily usage cap Apple has not published a specific number for.
- Samsung: Generative Edit and comparable image-generation tools are cloud-only by Samsung's own description; enabling "Process data only on device" will disable them rather than run them locally.
- Google: Gemini Nano access is currently limited to specific supported devices and is explicitly labeled experimental for developers through the Google AI Edge SDK — production apps use the more limited ML Kit GenAI APIs instead.
None of the three companies claims their on-device model replaces the cloud entirely. The pitch in every case is narrower than "AI that never leaves your phone" — it's "routine tasks stay local, complex ones still need a server, and here is how much control you get over that line."
Why every chipmaker is racing to build a bigger NPU
On-device AI only works if the phone's chip can run the model fast enough to feel instant, which is why NPU performance has become a headline spec. Qualcomm's own product brief for the Snapdragon 8 Elite Gen 5 — the chip that powers flagship Android phones from Samsung, Xiaomi, OnePlus, and others through 2026 — states the Hexagon NPU is 37% faster than the previous generation, with the entire pitch built around "continuous on-device learning and real-time sensing" where "user data stays on device."
That 37% figure isn't a coincidence of timing. Every step up in on-device model quality demands more NPU throughput to keep response times acceptable, and every hardware generation exists partly to keep pace with what the software side wants to run locally. The privacy pitch and the hardware upgrade cycle are the same business decision viewed from two different departments.
Frequently asked questions
Does on-device AI mean my data never leaves my phone?
Not necessarily. It means simple requests are processed locally by default in most implementations, but each of the three platforms covered here routes some requests to a company's servers — automatically in Apple's case, based on a device setting in Samsung's, and per-app in Google's.
Is on-device AI less capable than cloud AI?
Yes, by design. Apple, Google, and Samsung's own documentation all describe on-device models as deliberately smaller and less capable than their cloud counterparts, because they have to run within a phone's power and memory budget rather than a data center's.
Why do phone makers keep advertising bigger NPUs if the AI itself hasn't changed much?
Because a faster NPU is what lets a phone run the same or a slightly larger on-device model without a noticeable delay. Qualcomm's official product materials tie NPU speed gains directly to on-device AI features, not to gaming or general performance.
Can I turn off cloud processing entirely on any of these platforms?
Samsung is the only one of the three with an explicit, single toggle for this — "Process data only on device" in Galaxy AI settings — though Samsung states this will disable cloud-only features like Generative Edit rather than run them locally. Apple and Google do not offer an equivalent system-wide switch.
Last verified: 13 September 2026. This explainer draws on Apple, Google, Samsung, and Qualcomm's own published developer and security documentation; feature availability and behavior are subject to change with future software updates.
Sources
- Apple Machine Learning Research — Introducing the Third Generation of Apple's Foundation Models
- Apple Security Research — Private Cloud Compute: A new frontier for AI privacy in the cloud
- Apple Developer — WWDC26: Build with the new Apple Foundation Model on Private Cloud Compute
- Android Developers — Gemini Nano
- Android Developers Blog — An introduction to privacy and safety for Gemini Nano
- Samsung Newsroom — How Galaxy AI Protects Privacy with Samsung Knox Vault
- Qualcomm — Snapdragon 8 Elite Gen 5 Product Brief