Browser AI Runtimes

Definition and scope

Local browser execution using WebAssembly, WebGPU, WebNN where available, workers, storage, model downloads, caching, and progressive fallback.

Responsibilities

• Feature-detect capabilities
• Download and verify model assets
• Schedule work off the main thread
• Manage storage and cache eviction
• Preserve user control over local and hosted paths

Inputs, outputs, and boundaries

The layer consumes artifacts or requests from the layer above and relies on services from the layer below. Its contract should define supported inputs, produced outputs, lifecycle, compatibility, resource ownership, and failure semantics.

Failure modes

• Unsupported API or adapter
• Large download or storage eviction
• Main-thread blocking
• Device memory pressure
• Privacy regression through fallback

Implementation guidance

• Never assume uniform browser or hardware support.
• Use progressive enhancement and a server fallback only under explicit policy.
• Keep model size, cache, and update behavior visible to users.

Metrics

Measure the layer with workload-appropriate objectives. Avoid comparing unrelated categories or publishing unqualified performance numbers.

Capability negotiation

A browser runtime should negotiate capability before downloading a large model. Detection includes API availability, adapter limits, supported data types, storage quota, cross-origin isolation where required, worker support, and a practical memory budget. Capability checks should produce a stable execution profile that the application can use for model selection and user messaging.

Model download is part of the runtime lifecycle. Use versioned, integrity-checked artifacts; separate the application release from the model release; avoid filling storage without consent; and support cancellation and cleanup. Progressive fallback can move from an accelerator to WebAssembly, to a smaller local model, to a hosted endpoint, or to a non-AI workflow. The fallback must preserve the same data-classification and user-consent rules.

Security and lifecycle

• Keep untrusted model output outside HTML and command execution paths.
• Apply a restrictive content-security policy compatible with required workers and model assets.
• Version cache keys and remove superseded model packages.
• Do not expose provider credentials in browser code.
• Define what local storage contains, how it is cleared, and whether it synchronizes.
• Measure first-use download, warm startup, inference latency, memory pressure, and fallback rate.