Apple Intelligence On-Device: How FoundationModels Power Private Analysis
A technical deep-dive into Apple's Foundation Models framework and how on-device inference keeps your email analysis private by design — no servers, no cloud, no exposure.
| Property | Apple FoundationModels (iOS 26+) | Cloud LLM inference |
|---|---|---|
| Compute location | On the user's device (Apple Neural Engine + CPU + GPU) | Vendor's cloud GPUs |
| Model weights location | On the device, shipped with iOS | On vendor servers |
| Prompt leaves the device? | No | Yes — sent to the vendor's API |
| Network required for inference | No | Yes |
| Vendor can log prompts | No — no network path exists | Yes — by design |
| Latency per inference | Local (seconds) | Network round-trip + queue + inference |
When we say Veilsort runs "on-device," we are referring to Apple's Foundation Models framework — a system that runs large language models directly on the Neural Engine of A17 Pro and M-series chips. No cloud round-trip, no server inference, no data transmission. Here is how that works under the hood and why it matters for email privacy.
What Are Apple Foundation Models?
Apple introduced Foundation Models at WWDC 2024 as part of Apple Intelligence — a suite of on-device language models optimized for Apple Silicon. Unlike cloud-based models (GPT-4, Claude) that run on remote GPU clusters, Apple's Foundation Models are compiled for the Neural Engine — a specialized coprocessor on every recent iPhone that performs matrix operations at high throughput with extremely low power consumption. Apple's documentation describes these models as "designed for everyday tasks" with "personal context awareness" while "processing data entirely on-device."
Model weights are downloaded once from the App Store (Apple-vetted and code-signed). Inference runs in a dedicated memory region on the Neural Engine, isolated from the application processor. The model has no network capability — it reads text from a memory buffer and writes structured output back. No code path exists for the model to transmit, store, or log input data.
Where the Models Run
Foundation Models execute on Apple Silicon — the Neural Engine and GPU available on iPhone and iPad models that support Apple Intelligence, and on Apple Silicon Macs. The exact device set that supports Apple Intelligence is defined by iOS and Apple Silicon availability rather than by Veilsort; Veilsort requires iOS 26.0+ and an Apple Intelligence-capable device. When Apple Intelligence is not available — older hardware, iOS < 26, or the model not ready — Veilsort shows a graceful locked-state screen. It does not fall back to a network call. There is no network fallback by design.
How Inference Works Without a Network
When Veilsort receives a masked email for analysis, it constructs a prompt with the email text and instruction tokens, passes it to a Foundation Model through Apple's FoundationModels framework API (the on-device, Apple-bundled framework introduced with Apple Intelligence — not MLX, not Core ML, and not a custom-trained model), and receives a structured response — a Codable EmailAnalysis containing a summary, risk flags, deadlines, and a cautious suggested reply. No data is serialized for transport, no API call is made, no server round-trip occurs. On a supported device, the analysis completes quickly enough that the user perceives it as instant; the actual latency depends on input length and chunking, and we will publish measured timings when pre-launch validation is complete rather than assert a number here.
What Foundation Models Do Not Do
Foundation Models running on-device do not phone home, do not log prompts, do not send telemetry, and do not have any mechanism to access the network. Apple's WWDC 2024 session "Introducing Apple's On-Device Foundation Models" explicitly states: "All processing stays on the user's device. The models have no network capability and cannot transmit data." This is not a configuration option — it is a property of the runtime. Veilsort relies on this property as the foundation of its privacy guarantee.
Why This Matters for Email Analysis
The privacy benefit of on-device Foundation Models is not a feature that can be switched off or a policy that can be changed. It is architectural. Because the model has no networking capability, there is no code path that can accidentally or intentionally transmit your email content. As Google's Site Reliability Engineering principles note, "the most reliable systems are those where safety properties are enforced by the system design, not by operational controls." On-device Foundation Models enforce email privacy at the hardware level — the strongest possible guarantee.
Reference: Apple Developer Documentation — Foundation Models Framework. WWDC 2024 Session — "Introducing Apple's On-Device Foundation Models." Apple Platform Security Guide — Neural Engine Security.