NVIDIA donates Agent Substrate to the CNCF Sandbox: idle agent sessions are snapshotted with memory and filesystem, at 10x the density of standard containers
Agents spend most of their time waiting, while Kubernetes was built for long-running services. How Agent Substrate serves many sessions on a small pool of warm Pods.
// Key points
- Agent Substrate is an open-source, zero-trust runtime for sandboxed agents, now donated to the CNCF Sandbox; it multiplexes many sessions over a small pool of warm Kubernetes worker Pods, at what NVIDIA says is 10x the density of standard containers.
- When a session goes idle it is snapshotted out of its gVisor or microVM sandbox, memory and filesystem included, and its worker is freed; the next request restores it onto any free worker without going through the API server, scheduler or kubelet.
- It targets three workloads: coding assistants alternating between model calls and repository work, bursty reinforcement learning environments, and short-lived GPU sandboxes for compiling and benchmarking.
- Upstream additions include request parking, which queues a turn when every worker is busy instead of returning an error, and integration with NVIDIA Dynamo, whose router uses session identity and KV cache location to keep related calls on workers that already hold the context.
Builder's takeTeams running their own agent platform usually face a choice between one warm Pod per session, which is expensive, and on-demand Pods, which cold-start on every step. If your sessions look like "run 10 ms, wait minutes", I would load-test Agent Substrate on a staging cluster for density and restore latency before building more of our own session pool. Next to AWS Nitro Sandboxes on 11-09, both isolation and density are becoming standard parts.
// Background · from #AI agents
Full timeline →- Nov 11 Amazon Bedrock AgentCore launches a Web Search API: ranked web results in one call at $7 per 1,000 queries
- Nov 11 ElastiCache Serverless for Valkey adds vector, full-text and hybrid search, with up to 99% vector recall and writes searchable within microseconds
- Nov 10 AWS launches Nitro Sandboxes: each EKS pod, ECS task or lightweight VM gets its own hardware-isolated VM, enabled with a one-line change at no additional charge