Mobile AI architecture: seven patterns and their trade-offs

A mobile AI feature needs decisions about where requests run, what data crosses a boundary and what happens when the phone disconnects. The seven patterns below describe useful options rather than a fixed industry taxonomy. Some concern inference, others orchestration or interaction, so a product may combine them. None is an automatic answer to privacy, regulation or response time.

Pattern 1: backend-mediated cloud inference

The app calls an authenticated backend, which checks access, prepares the request and contacts the model service. This suits tasks that need server-held information or a cloud model’s capabilities. Keep provider credentials on the backend, set usage limits and define cancellation and retry behaviour. Measure the whole interaction, including retrieval and network time; there is no universal latency or cost ranking for this pattern.

Pattern 2: permission-aware caching

Cache reusable information or model results where the task permits it. Define what makes a request equivalent, how sources invalidate an answer and whether the result is personal. A cache key must respect user and organisation access boundaries. Test stale answers and account changes before claiming savings. Measure the cache-hit rate and actual provider usage; a shared FAQ and a personalised assistant have very different reuse opportunities.

Pattern 3: local routing with cloud fallback

A local rule or model can recognise a supported intent, prepare an input or complete a narrow task; a backend handles requests outside that boundary. For example, route a known command locally but ask the server to search approved business information. Specify how confidence, missing data and network loss change the route. Evaluate both paths and give users a clear explanation when a task cannot continue.

Pattern 4: on-device inference

Run a supported model on the device for a bounded task, such as rewriting or summarisation. Availability depends on the OS, hardware, language and API. Apple Foundation Models and Android’s Gemini Nano guidance describe platform-managed options. Check actual device availability at runtime and define a fallback. Local execution does not by itself guarantee a suitable answer or establish how logs and synchronised data are handled.

Pattern 5: local personalisation and model updates

An app can store preferences locally or adapt a narrow feature without distributing inference across users’ devices. Federated learning is a separate approach to training or evaluating models using decentralised data, as TensorFlow Federated’s documentation explains. It is not an offline inference pattern or a blanket privacy guarantee. Treat shared model updates as specialist work with its own evaluation, security and operational requirements.

Pattern 6: server jobs with mobile status

For a long-running agent task, submit an authorised job to the backend and let the app show progress, cancellation and results. Do not assume an iOS system extension or Android service can run an unrestricted agent indefinitely. Apple’s Background Tasks guidance and Android’s background-task guidance define platform-specific mechanisms and limits. Design reconnection and notification around those limits; the server job must also enforce permissions, expiry and duplicate prevention.

Pattern 7: streaming voice interaction

A voice feature may stream audio through a speech-to-text, model and text-to-speech pipeline, or use a provider’s real-time audio interface. Design microphone permissions, turn-taking, interruption, transcription review and reconnect behaviour together. Measure time to first useful response under realistic networks and accents. A single latency threshold cannot establish a good experience, and streaming does not mean the model runs inside the phone.

Choose by task and failure mode

Ask what must work offline, what information may leave the device and what result is acceptable when a component fails. Offline support requires local capability for that specific task; federated learning does not provide it. Sensitive or regulated data needs appropriate review of processing, storage, access and contracts. Neither on-device execution nor a provider agreement alone establishes that the complete app meets its obligations.

Test beyond the model response

Include permission denial, unavailable models, large inputs, weak networks, interrupted audio and a signed-out user. On devices, test memory, battery, thermal behaviour and OS support. For local and cloud state, define the source of truth and conflict handling. For tools that change records, test authorisation, confirmation, duplicate requests and recovery. Keep a representative evaluation set alongside device and integration tests.

Plan the operating costs

Cloud costs depend on request volume, input and output size, retrieval, retries and any tools used. Local execution introduces device-support and model-management work, even if it avoids some per-request charges. Caching only helps where safe reuse occurs. Record assumptions, instrument actual consumption and set limits; compare architectures using the task your app performs rather than a promised percentage saving.

Review platform availability before committing

Platform-managed AI can reduce some model-distribution work, but availability and supported tasks vary. Check the current official documentation, test runtime availability and maintain a device-support matrix for the feature. A custom model route has different download, update and compatibility responsibilities. Label what the app can do locally, what requires connectivity and what happens on unsupported devices.

Finish with a decision record for each feature: execution location, allowed data, supported devices, source permissions, completion criteria, recovery path and operating owner. A local suggestion, a cloud-backed document answer and a long-running server task may belong in the same app. Choose and test each route explicitly instead of using one agent label for every interaction.