AI Runtime Resources

OpenAI’s Agents SDK is a Python framework for building agentic workflows with agents, tools, handoffs, guardrails, sessions, tracing, and human-in-the-loop patterns. It is useful when an application wants managed agent execution above direct model calls, not merely a single completion endpoint.

• Tools
• Guardrails
• Tracing

LangGraph focuses on low-level orchestration for long-running, stateful agent and workflow execution. It is relevant when the runtime must persist state, resume after failures, stream intermediate events, support human interruptions, and keep application logic separate from one-shot model calls.

• Durable execution
• State
• Human review

Google ADK provides documentation and tooling for building agent applications around models, tools, and structured execution flows. On a runtime resources page, it belongs in the agent-development category rather than the low-level inference-engine category, because it addresses application behavior above model serving.

• Agents
• Tools
• Development

Semantic Kernel is application middleware for composing AI services, prompts, plugins, memory-oriented patterns, and orchestration logic in enterprise applications. It can help standardize how product code invokes model and tool capabilities, but production runtime boundaries still require external policy, identity, logging, and deployment controls.

• Planning
• Plugins
• Enterprise apps

The Vercel AI SDK is aimed at web application developers who need streaming model responses, provider abstraction, UI integration, and server-side helpers. It is not a complete runtime by itself; it is best viewed as middleware that connects product interfaces to model and tool execution paths.

• Streaming
• Web apps
• Model providers

Assistant UI’s runtime guide helps developers choose how a chat or assistant interface should connect to backend execution. It is useful for understanding front-end runtime adapters, message state, streaming UI behavior, and integration boundaries between interface components and server-side agent logic.

• UI state
• Frontend
• Adapters

AgentScope Runtime for Java is positioned around agent deployment and tool sandboxing for Java-oriented ecosystems. It is relevant when runtime design must isolate tools, manage sessions, expose observable execution, and integrate with Java agent frameworks rather than relying solely on Python-first orchestration stacks.

• Java
• Sandboxing
• Tools

MCP standardizes how AI applications connect to external tools, data sources, prompts, and workflows. For runtime architecture, it is most useful at the tool and context boundary: discovery, permissioning, connector design, and controlled access to systems outside the model provider.

• Tools
• Data sources
• Discovery

A2A addresses communication and collaboration between agents rather than connecting one AI application to tools. It belongs in runtime planning when multiple agents, specialized services, or organizations need typed handoffs, task status exchange, and clearer boundaries for cross-agent collaboration.

• Agent collaboration
• Interoperability
• Handoffs

OpenAPI defines machine-readable HTTP API contracts. In AI runtimes, it is useful for turning APIs into typed tools, documenting authentication and parameters, validating request and response shapes, and separating tool execution contracts from prompt text or ad hoc natural-language instructions.

• HTTP APIs
• Schemas
• Tool contracts

JSON Schema defines how JSON documents can be validated against a formal schema. Runtime designers use it for tool inputs, structured model outputs, response envelopes, trace events, configuration, and policy checks that should fail deterministically before side effects occur.

• Validation
• Structured output
• Schemas

vLLM is an LLM serving engine focused on high-throughput generation, online serving, batching, prefix caching, structured outputs, and distributed deployment patterns. It is a model execution and serving component; it still needs surrounding runtime controls for identity, tool authorization, memory policy, and product workflow.

• LLM serving
• Batching
• KV cache

Ollama helps developers run and manage local models through a convenient local service and CLI-oriented workflow. It is useful for desktop, prototype, and private local inference patterns, but production runtime design still needs routing policy, observability, authorization, and deployment controls around it.

• Local models
• Developer workstation
• Model management

ONNX Runtime executes ONNX model graphs across CPU, GPU, mobile, browser, and accelerator backends through execution-provider abstractions. It is relevant when portability, graph optimization, hardware delegation, and model-format interoperability matter more than a single provider-specific serving stack.

• ONNX
• Execution providers
• Portability

Triton Inference Server provides multi-framework model serving with HTTP and gRPC APIs, model repositories, scheduling, dynamic batching, and observability integrations. It is a serving platform for the execution layer, not a full agent runtime or policy system by itself.

• Model server
• gRPC
• Dynamic batching

KServe provides Kubernetes-native abstractions for deploying, scaling, and operating model inference services. It is most relevant when runtime teams need platform-level serving resources, traffic management, standardized inference endpoints, and integration with Kubernetes operations rather than a standalone local inference engine.

• Kubernetes
• Autoscaling
• Rollouts

Ray Serve is a scalable serving library for composing Python applications, model deployments, and distributed services on Ray. It fits runtime architectures that need programmable request handling, autoscaling, composition of multiple model or preprocessing steps, and deployment across a distributed application substrate.

• Python services
• Autoscaling
• Composition

OpenTelemetry’s Generative AI semantic conventions define shared telemetry vocabulary for model requests, providers, usage, and related signals. The formerly linked documentation page now points to the maintained semantic-conventions repository, so ARuntime uses the current repository location for link accuracy.

• Tracing
• Metrics
• Semantic conventions

Langfuse provides observability, tracing, prompt management, evaluations, datasets, and production monitoring for LLM applications. It is useful when teams need to analyze runtime behavior across model calls, tool steps, latency, cost, quality, and prompt or model changes.

• Tracing
• Evaluation
• Prompt management

Arize Phoenix is an observability and evaluation system for AI applications, including tracing, datasets, experiments, and LLM evaluation workflows. It is relevant for runtime teams that need to debug requests, compare outputs, inspect retrieval behavior, and connect offline evaluations with production traces.

• Tracing
• Evaluation
• Datasets

The OWASP GenAI Security Project publishes open guidance, threat lists, governance resources, and educational material for securing generative and agentic AI systems. It is useful for mapping runtime controls to prompt injection, excessive agency, data exposure, memory risks, and operational governance.

• Prompt injection
• Agent security
• Controls

The NIST AI Risk Management Framework provides a structured vocabulary for identifying, measuring, managing, and governing AI risks. Runtime teams can use it to connect engineering controls with organizational risk processes, documentation, monitoring, incident handling, and accountability expectations.

• Risk
• Governance
• Assurance

MITRE ATLAS catalogs adversary tactics and techniques against AI-enabled systems. It helps runtime and security teams think about threat modeling, incident response, supply-chain exposure, model behavior manipulation, data attacks, and controls that should exist outside prompt-level instructions.

• Threat modeling
• Adversary behavior
• Incident analysis

The MCP security guidance covers authorization risks, confused-deputy problems, token handling, server-side request forgery, session hijacking, local-server compromise, and scope minimization. It is directly relevant to runtime designs that expose external tools or data sources through MCP connectors.

• MCP
• Authorization
• Tool access