The network drops just as a maintenance technician asks a tablet, “What should I do about this pump?” A useful assistant still opens the downloaded inspection record. A dangerous one converts an incomplete description into permission to restart equipment. Both can run locally. Locality tells us where computation happens; it does not tell us what the system knows or what it may do.
Consider a hypothetical maintenance assistant for a building whose lower floors have unreliable connectivity. The tablet holds selected equipment records, inspection forms, and a small language model; an optional workstation on the building network hosts a larger model. A cloud model is available for approved requests. Nothing in this example is a reported deployment or benchmark. It is a reference design for deciding what each tier receives, returns, and refuses.
Start with the part that must work when every connection disappears. The technician can open downloaded records, enter observations, and save an inspection locally. Android’s offline-first architecture guidance makes a useful distinction here: critical functionality remains available without internet access, while local and network data can temporarily differ. The assistant should show when a record was last synchronized. “Available offline” must not quietly become “known to be current.”
The first rung needs no model. An explicit tap on “Open inspection” resolves a known equipment identifier and opens the corresponding local form. A deterministic handler validates the identifier, the requested operation, and the technician’s access before proceeding. It can return a precise result even when a language model is unavailable. We should keep that path because asking a model to reinterpret an already explicit command adds another opportunity to misunderstand it.
The tiny model earns its place when the request arrives as language. “Pull up yesterday’s inspection for pump seven” becomes a proposed typed action with an operation, equipment identifier, and date. The application checks those fields against the local inventory and asks a question if “pump seven” matches two assets. Its output is an interpretation to validate, not an instruction to execute blindly. If the request already fits this narrow contract, there is no reason to send it through every larger model.
The next rung handles interpretation that exceeds that contract. Suppose the technician asks whether today’s vibration note resembles a previous incident. A local general model can compare retrieved inspection excerpts and explain the resemblance, provided the relevant records are available and fit its context. It should return the records it relied on, their timestamps, and the uncertainty that remains. A fluent explanation without those references leaves the technician unable to distinguish a documented pattern from a plausible story.
“Local” needs a physical definition at this point. A model on the tablet survives the loss of both internet and the building network, subject to the device’s own resource limits. A model on the workstation survives an internet outage only while the tablet can still reach that workstation. If that connection fails, the interface should say that comparison is unavailable and preserve the observation for later review. It should not replace the missing comparison with an unmarked guess from a smaller model.
The cloud rung begins with an unresolved question and a proposed evidence packet. Perhaps the local comparison cannot explain an unusual combination of symptoms, and the technician wants a broader analysis. The application assembles the minimum relevant excerpts, removes prohibited fields, and presents the outgoing material for approval. In this example, approval applies to that packet and purpose. It does not authorize uploading the complete maintenance archive or treating future requests as preapproved.
Redaction needs more care than deleting names. An equipment location, incident narrative, or attached image may still expose information that the organization does not allow outside the building. The packet therefore needs an allowlist of permitted fields, inspection of free text, and a visible account of what was removed. If removing context makes the question misleading, the assistant should explain that limitation. Some questions belong with a person who can consult the private record rather than with a remote model receiving an impoverished version.
Each handoff should carry enough evidence for the receiving tier to know the task it has actually been given. For this assistant, I would include the original request, validated asset identity, relevant record versions, selected excerpts, the unresolved question, and the remaining time budget. The cloud return would include its recommendation and references back to those supplied excerpts. The application would preserve which claims came from records and which were model interpretations. Sending only “the previous model was unsure” throws away the information needed to assess the answer.
The final rung is a person, especially when the request changes from understanding a fault to acting on equipment. A recommendation to inspect a seal is different from a command to restart a pump serving an occupied building. The latter requires the appropriate operator, current conditions, and the applicable procedure; a more capable model cannot supply that authority by sounding certain. OWASP’s authorization guidance recommends denying access by default and validating permissions on every request. The execution boundary needs those checks regardless of which model proposed the action.
Latency belongs in the routing decision before the expensive call begins. As illustrative design targets, this tablet might acknowledge an input within 200 milliseconds, allow two seconds for narrow interpretation, and offer a cancellable background task when deeper analysis exceeds ten seconds. These are proposed budgets, not measured capabilities or safety guarantees. Tests on the actual devices would determine whether they are achievable. A slow local model should not trigger an automatic cloud upload merely because a timer expired.
Caches need equally explicit boundaries. A downloaded manual can remain useful when the network fails, provided its version and freshness are visible. A cached explanation needs a key that includes the question, asset, source versions, model configuration, and relevant access context; it should be invalidated when those inputs change. Permission to perform an operation must be checked again rather than recovered from a previous answer. Otherwise a latency optimization can quietly reuse yesterday’s authority or apply one machine’s explanation to another.
Connectivity loss becomes most consequential after an operation has been submitted. A timeout may mean that the request never arrived, or that it succeeded and the response was lost. AWS’s discussion of timeouts and retries explicitly warns that failure does not establish that side effects did not occur. The assistant should retain the operation identifier, mark the outcome unknown, and reconcile it with the receiving system before retrying. Where the receiver supports idempotency, the retry must preserve the same logical operation identifier.
That leaves a practical test for the proposed ladder. Disconnect the tablet before retrieval, during interpretation, before an approved cloud request, and after submitting an operation. Check that each interruption preserves the technician’s work, accurately reports what is unavailable or unknown, and keeps execution behind the required authority. Also check that a straightforward request never climbs the ladder unnecessarily. The architecture succeeds when losing a tier removes a capability the user can name, while leaving the remaining behavior understandable and dependable.
Subscribe to Signal Over Noise for practical analysis of AI agents and automation: what to test, which constraints matter, and when a tool is worth using. Bring the next idea to one bounded workflow and check the result before expanding it.
If this was useful, forward it to one engineer who needs less noise in their feed.