Browser AI Application

Problem

Browser inference can keep data near the user and reduce server demand, but browser, operating-system, device, driver, memory, storage, and API support vary. Large model downloads can harm performance and consume storage; cached artifacts require integrity and version management; web pages remain exposed to supply-chain, origin, extension, and cross-site risks. The runtime must deliver a useful fallback instead of assuming one accelerator API is universally available.

Why a runtime layer is needed

The browser application runtime selects among WebGPU, WebNN where supported, WebAssembly, worker-based CPU execution, or a governed server route. It owns capability detection, model-manifest verification, origin storage, progressive download, cache eviction, privacy labeling, cancellation, and fallback. The graph or inference runtime executes the model; the application runtime handles the web lifecycle and user-visible execution contract.

Reference architecture

A static application shell and server-rendered fallback remain usable before model code loads or when JavaScript is unavailable.
A capability probe runs after user interaction and records supported APIs, limits, cross-origin isolation state, memory pressure indicators, and storage availability.
A model manifest identifies version, files, hashes, size, required operations, preferred backend, minimum capability, license, and fallback routes.
Model artifacts download incrementally over HTTPS, are integrity-checked, and enter an origin-scoped cache with explicit quota and eviction behavior.
Inference runs in a worker when practical so long operations do not block the interface. Cancellation, progress, and degraded-mode messages are accessible.
Inputs stay local unless a disclosed server fallback is selected. The fallback request is minimized, classified, and processed under a separate retention policy.
Service-worker behavior, cache updates, and model activation are versioned to avoid mixing incompatible application, tokenizer, and weight versions.

Request flow

Render a functional application shell and obtain user intent before loading expensive model assets.
Detect capabilities and choose a supported local backend/model pair or a declared fallback.
Check cached manifest and artifact hashes; download missing chunks with progress and cancellation.
Prepare input locally, apply size and classification limits, and execute in a worker or isolated context.
Validate output and display whether processing occurred on-device or through a server route.
Apply the cache and local-storage policy; do not persist inputs merely because the model is cached.
Record coarse performance and error signals without collecting sensitive prompt text by default.

Contracts

Model manifest: application compatibility range, model/tokenizer versions, chunk URLs, integrity hashes, sizes, operation requirements, and backend priorities.
Capability result: supported APIs/features, tested limits, storage estimate, preferred backend, and reason for fallback without creating an invasive device fingerprint.
Fallback policy: user disclosure, input minimization, prohibited data, endpoint, retention, timeout, and local retry behavior.
Cache policy: origin namespace, maximum size, version activation, eviction, stale cleanup, and private-browsing behavior.

Failure modes and recovery

Detection, behavior, retry, idempotency, recovery, and evidence
Failure	Detection	User-visible behavior	Retry and idempotency	Recovery and evidence
Required browser capability absent	Feature detection or small conformance probe fails.	Use a tested CPU/server fallback or explain the unsupported feature.	Do not repeatedly probe a known unsupported path during the session. No model side effect occurs before backend selection.	Persist only a short-lived capability decision and allow recheck after browser updates. Evidence: Capability category, selected fallback, and user disclosure.
Model download interrupted	Fetch abort, network loss, integrity mismatch, or quota error.	Preserve application usability and offer resume or smaller model.	Resume only verified chunks; redownload integrity failures. Chunks are content-addressed and activated atomically.	Discard corrupt chunks, free quota, and retain the last compatible version. Evidence: Manifest version, chunk hashes, bytes transferred, and recovery.
GPU device lost	WebGPU device-lost signal or backend execution error.	Cancel the operation and offer CPU or server fallback.	Retry once after recreating the device when safe; otherwise change backend. Inference has no external side effect.	Release resources, reset backend, and avoid an infinite restart loop. Evidence: Backend, device-loss reason, retry, and fallback outcome.
Origin storage unavailable	Private mode, quota, eviction, permission, or storage API failure.	Run without persistent cache when feasible and explain repeated download implications.	Do not spin on failed writes. Cache activation uses versioned keys and transactions where available.	Use in-memory execution or smaller assets; clear partial cache entries. Evidence: Storage mode, quota estimate, failed keys, and selected mode.
Server fallback denied or unavailable	Privacy policy, user choice, network, or server health blocks fallback.	Keep the input local and return a bounded inability message or local reduced result.	Retry only within the deadline and policy; never bypass denial. No server request is sent before policy and user-state checks.	Queue nothing sensitive by default; offer an explicit later retry. Evidence: Fallback decision, endpoint health category, and final execution location.

Security considerations

Use a strict origin and dependency policy; browser-local execution still depends on application scripts and model artifacts delivered by the site.
Verify model manifests and artifact hashes before activation, and avoid evaluating model-provided or downloaded code.
Do not derive an invasive device fingerprint from capability detection; collect only what backend selection needs.
Keep input, output, caches, and diagnostics origin-scoped and avoid storing sensitive content in service-worker caches or analytics.
Separate model artifact caching from user data storage and provide clear controls to remove both.
Apply the same output encoding and untrusted-content rules to locally generated output as to server-generated output.

Observability

Measure backend selection, model version, download/cache state, time to usable model, inference latency, device-loss events, and fallback without raw inputs.
Use performance marks and worker messages that do not expose sensitive content.
Track integrity failures, cache activation failures, storage quota, and browser-family compatibility trends.
Provide a user-exportable diagnostic summary for local failures instead of automatically uploading detailed device data.

Evaluation

Test current major browsers, private modes, low-memory devices, throttled networks, offline states, cache eviction, and reduced storage.
Verify numerical or task-level output across WebGPU, WebNN, WebAssembly/CPU, and server fallback within declared tolerances.
Test accessibility during download, progress, cancellation, errors, and fallback selection.
Use network inspection to verify that local-only modes do not transmit inputs.

Metrics

Local backend eligibility and successful activation rate
Initial and repeat-load model readiness time
Artifact bytes, cache hit rate, integrity failures, and quota errors
Inference latency and cancellation responsiveness by backend
Device-loss and backend-fallback rate
Server-fallback rate and input-egress compliance
Core Web Vitals impact of model loading and execution

Implementation checklist

Keep a server-rendered or native web fallback independent of model JavaScript.
Use feature detection plus a small execution probe, not user-agent assumptions.
Version and integrity-check application, tokenizer, model, and cache manifests together.
Run heavy inference off the main thread and make progress and cancellation accessible.
Separate local input storage from model caching and disclose execution location.
Test offline, private mode, low quota, device loss, and unsupported API paths.

Sources

WebGPU

World Wide Web Consortium · W3C specification · 2026-06-23T00:00:00Z

Browser GPU API and execution model relevant to local inference and capability detection.
Web Neural Network API

World Wide Web Consortium · W3C specification · 2026-06-23T00:00:00Z

Browser neural-network graph API, supported operations, and execution context.
OWASP Top 10 for LLM Applications 2025

OWASP Foundation · Security guidance · 2026-06-23T00:00:00Z

Threat categories for prompt injection, sensitive information disclosure, excessive agency, and insecure output handling.
Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile

National Institute of Standards and Technology · Government framework · 2026-06-23T00:00:00Z

Risk identification, measurement, governance, and lifecycle controls for generative AI systems.

Find runtime definitions and implementation guidance