Managed Agent Runtime

also known as Agent Loop as a Service, Managed Agent Loop, Serverless Agent Runtime

Offer the agent loop itself as a managed cloud primitive so a caller supplies a model, system prompt, and tools and the platform runs the orchestration in an isolated, session-scoped runtime.

Context

An organisation is shipping several agent products and every team has hand-rolled the same orchestration loop: the reason-act cycle, tool dispatch, session state, per-session isolation, identity for outbound calls, retries, and observability. Each loop is operated separately, drifts in behaviour, and carries its own on-call burden, while the cloud now exposes the loop as a service alongside memory, identity, and tracing.

Problem

Re-implementing and operating the agent loop per team multiplies effort and risk. Each bespoke loop must solve session isolation, credential handling, resumability, and tracing on its own, and any one of them done weakly becomes a production incident. Sessions that share a process leak state into one another; outbound tool calls reach for ambient host credentials; a crash loses in-flight work because there was no checkpoint; and there is no uniform trace to debug across products. The loop is undifferentiated infrastructure that every team rebuilds and operates instead of consuming.

Forces

A bespoke loop gives full control over orchestration but multiplies operational surface across teams.
Session-scoped isolation needs a sandbox per session, which adds cold-start latency.
A managed runtime standardises identity, memory, and tracing but ties the deployment to a vendor contract.
Built-in observability is uniform across products but constrained to what the platform emits.

Example

A company runs a coding agent, a research agent, and a support agent, each on a separately operated loop, and a state leak between two concurrent support sessions causes one user to see another's data. The teams move to a managed agent runtime: each defines its model, system prompt, and tools and invokes a single API, and the platform runs every session in its own microVM with built-in identity and tracing. The cross-session leak becomes structurally impossible and the three teams stop carrying three on-call rotations for the same loop.

Diagram

flowchart TD Caller[Caller: model + system prompt + tools] --> Invoke[Managed runtime: single invoke API] Invoke --> Session[Session-scoped sandbox<br/>microVM: fs + shell] Session --> Loop[Managed reason-act loop] Loop --> Mem[Built-in memory] Loop --> Id[Managed identity:<br/>scoped credentials] Loop --> Obs[Observability: uniform trace] Loop --> Result[Result to caller]

Solution

Therefore:

The platform exposes a single invoke endpoint that takes a model reference, a system prompt, and a tool set, then runs the full reason-act loop on the caller's behalf. Each session executes inside its own isolated sandbox, commonly a microVM with filesystem and shell access, so concurrent sessions never share state. The runtime wires in managed memory for short- and long-term context, a managed identity service that mints scoped credentials for outbound tool calls, and observability that emits a uniform trace of every step. The caller owns prompt, tools, and policy; the platform owns the loop, isolation, and operations.

What this pattern forbids. Callers cannot run the orchestration loop in their own process; the loop runs only inside the platform's session-scoped sandbox, and each session is isolated rather than sharing a runtime.

The smaller patterns that complete this one —

usesReAct★★— Interleave a single thought, a single tool call, and a single observation per step so the agent reasons over fresh evidence.
usesSandbox Isolation★★— Run agent-emitted code or actions in a contained environment with restricted filesystem, network, and process privileges.

And the patterns that stand alongside it, or against it —

complementsSession Isolation★★— Keep one user's session state and memory unreachable from another user's agent.
complementsAgent Resumption★★— Persist agent execution state so a long-running run survives restarts, deploys, or user disconnects.