Enterprise Knowledge Assistant

Problem

A useful enterprise assistant must search heterogeneous repositories without turning a broad natural-language question into broad data access. The central design problem is not only retrieval quality. It is preserving the caller’s identity, tenant, purpose, classification limits, source freshness, and citation requirements across retrieval, model execution, and response assembly. A generic chat interface that retrieves first and checks policy later can disclose restricted material even when the final answer is filtered, because sensitive content may already have entered prompts, logs, caches, or traces.

Why a runtime layer is needed

The agentic application runtime establishes the request boundary and owns authorization-aware context assembly, model routing, citation policy, memory decisions, and evidence. The inference and serving layers still own token generation, batching, and model health. Separating these responsibilities lets the system optimize inference independently while keeping identity, data access, and answer provenance enforceable at the application boundary.

Reference architecture

An admission service validates the runtime-request contract, resolves actor and tenant identity, and assigns a correlation identifier before any retrieval occurs.
A policy decision point translates identity, purpose, risk level, and data classification into source filters and retrieval permissions.
Retrieval connectors query only approved repositories and return document identifiers, version metadata, access-control labels, timestamps, and bounded excerpts rather than unrestricted document bodies.
A context assembler deduplicates passages, enforces a token and sensitivity budget, records context hashes, and separates trusted system instructions from retrieved text.
A model router selects a deployment whose residency, retention, capability, latency, and cost properties satisfy the request constraints.
The inference service generates a structured answer containing claims and source references. A validator rejects unsupported citations, malformed output, or answers that exceed disclosure policy.
The evidence plane stores source references, policy decisions, model-route metadata, redaction events, and evaluation results without assuming that raw prompts may be retained.

Request flow

Validate the request, deadline, identity, tenant, task type, risk, and data classification.
Resolve effective permissions and create source-specific retrieval filters.
Retrieve candidate passages and preserve source/version/access metadata.
Apply classification, freshness, duplication, and token-budget filters before prompt assembly.
Route to an approved model deployment and invoke inference with citation requirements encoded in the output contract.
Verify that each material claim maps to an allowed source and that citations resolve to the version used during generation.
Return the answer with citations; record a minimized evidence envelope and apply the configured memory policy.

Contracts

Runtime request: actor, tenant, purpose, riskLevel, dataClassification, contextPolicy, modelRouteConstraints, outputContract, retentionPolicy, and traceSettings.
Retriever tool: repository identifier, query schema, authorization reference, allowed classifications, timeout, result cap, and source metadata requirements.
Citation output contract: claim identifiers, source identifiers, source version, supporting span, confidence or support status, and unresolved-claim behavior.
Memory policy: no durable storage by default; explicit scope, expiration, minimization, and user-visible controls when memory is enabled.

Failure modes and recovery

Detection, behavior, retry, idempotency, recovery, and evidence
Failure	Detection	User-visible behavior	Retry and idempotency	Recovery and evidence
Authorization context missing	Identity or tenant resolution returns no authoritative subject or policy context.	Reject before retrieval and explain that access context could not be established.	Retry only after reauthentication or authoritative context recovery. No side effect is permitted before admission completes.	Invalidate partial context and restart admission. Evidence: Authentication outcome, policy version, and rejection reason.
Retrieval source unavailable	Connector timeout, circuit breaker, or stale index health signal.	Return a scoped partial-answer notice or fail closed when required sources are mandatory.	Retry read-only retrieval with bounded backoff within the request deadline. Retrieval calls are read-only and correlated by request ID.	Use an approved alternate source only when policy allows it. Evidence: Connector attempts, source health, selected fallback, and omitted scope.
Restricted passage returned	Post-retrieval classification does not match the allowed classification set.	Do not expose or send the passage to the model; continue only if enough allowed evidence remains.	Do not retry the same unrestricted query. Record the rejected result identifier, not its sensitive body.	Tighten filters, alert on repeated connector-policy mismatch, and invalidate affected caches. Evidence: Policy decision, classification labels, redaction action, and connector version.
Unsupported citation	Answer validator cannot map a claim to a retrieved source/version.	Regenerate once with stricter constraints or return an explicit unsupported statement.	One bounded regeneration is normally eligible; repeated failures stop. The response attempt is versioned under the same request and correlation IDs.	Remove unsupported claims or route to human review for high-impact use. Evidence: Claim-to-source validation results and final disposition.
Trace persistence unavailable	Evidence sink rejects or times out.	For high-risk requests fail closed; for lower-risk requests follow declared degraded-mode policy.	Retry asynchronously only when durable queueing is configured and disclosed. Evidence writes use deterministic record identifiers.	Persist to a bounded local spool or reject; never silently discard mandatory evidence. Evidence: Evidence outage state, spool location reference, and final persistence status.

Security considerations

Apply authorization before retrieval and again before presenting links or excerpts; do not assume that search-index access controls are sufficient.
Treat retrieved text as untrusted data. Delimit it from instructions and prevent it from changing tool, policy, or memory configuration.
Minimize prompt, cache, log, and trace content; prefer stable identifiers and hashes when raw text retention is not authorized.
Partition caches and vector indexes by tenant and policy context; include authorization-relevant dimensions in cache keys.
Require a documented hosted-fallback policy when requests may leave the organization’s controlled environment.
Keep source credentials in connector-specific secret scopes and never expose them to the model context.

Observability

Correlate admission, policy, retrieval, reranking, model inference, validation, and response spans without requiring raw prompt capture.
Record source counts by repository and classification, filtered-result counts, context-token use, model route, citation-validation outcomes, and cache behavior.
Separate infrastructure traces from evidence records: traces diagnose execution; evidence supports review of a specific business outcome.
Alert on cross-tenant cache keys, repeated classification mismatches, citation failure spikes, and unexplained source-coverage changes.

Evaluation

Evaluate retrieval recall only against access-filtered ground truth; unrestricted recall is not an acceptable target.
Measure citation correctness, claim support, refusal correctness, classification-boundary preservation, and stale-source behavior.
Maintain adversarial tests for prompt injection embedded in documents, conflicting source versions, revoked permissions, and poisoned metadata.
Run evaluations per model route and source connector version because either may change answer behavior.

Metrics

Authorized retrieval recall and precision
Citation precision and material-claim coverage
Unsupported-claim rate and correction rate
Cross-tenant or classification-boundary violations (target: zero)
End-to-end latency by admission, retrieval, inference, and validation phase
Context tokens, cache hit rate, and cost per accepted answer
Evidence completeness and mandatory-trace persistence rate

Implementation checklist

Define authoritative identity, tenant, purpose, and data-classification sources.
Enforce access controls in connectors and validate them again at context assembly.
Version retrievers, indexes, prompts, policies, model routes, and citation validators.
Specify unsupported-answer, stale-source, partial-source, and evidence-outage behavior.
Test revocation, tenant isolation, prompt injection in documents, and cache partitioning.
Expose source versions and correction channels to users without revealing restricted metadata.

Sources

Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile

National Institute of Standards and Technology · Government framework · 2026-06-23T00:00:00Z

Risk identification, measurement, governance, and lifecycle controls for generative AI systems.
OWASP Top 10 for LLM Applications 2025

OWASP Foundation · Security guidance · 2026-06-23T00:00:00Z

Threat categories for prompt injection, sensitive information disclosure, excessive agency, and insecure output handling.
OpenTelemetry Traces

OpenTelemetry · Official project documentation · 2026-06-23T00:00:00Z

Trace and span semantics used to correlate runtime, model, tool, and policy operations.
PROV-Overview: An Overview of the PROV Family of Documents

World Wide Web Consortium · W3C recommendation · 2026-06-23T00:00:00Z

A formal vocabulary for provenance relationships among entities, activities, and agents.

Find runtime definitions and implementation guidance