NVIDIA runs Dynamo 1.4.2 on the open-source AI Cluster Runtime: validated recipes pin about 18 Helm charts for GB200 and GB300 NVL72 on EKS
Self-hosted inference usually loses time in the layer below the model; this packages drivers, scheduling, metrics and ingress into recipes you can validate.
// Key points
- Running Dynamo on Kubernetes needs a lower layer of GPU drivers, device allocation, scheduling, node health, metrics and ingress; the open-source AI Cluster Runtime (AICR) packages it as validated recipes that pin component versions with compatibility constraints and validation checks.
- The AICR v1.0.0 recipe targets Amazon EKS for GB200 NVL72 and GB300 NVL72 with about 18 pinned Helm charts, including GPU Operator v26.7.1 and KAI Scheduler v0.16.9, plus the dynamo-platform chart at 1.4.2.
- Two routing paths: the Dynamo Frontend router for the simplest topology, or the Gateway API Inference Extension with Endpoint Picker for shared ingress, both KV-cache-aware; NATS is no longer installed by default.
- Five steps: snapshot the cluster, generate a recipe, render a bundle, deploy the runtime and validate before installing applications, via Helmfile, a plain Helm script or GitOps bundles for Argo CD and Flux. Dynamo 1.5 plans to replace the Go/CGO EPP bridge with a native Rust EPP.
Builder's takeMost product teams will never touch an NVL72, but pinning the version set and validating it is worth copying: self-hosted inference usually breaks on version drift between drivers, scheduler and gateway, not on the model. If you self-host open models, write the whole dependency set down as one replayable manifest and change one thing per upgrade. Read alongside the 10-27 Cloudflare cross-request KV cache leak: once you route on KV cache, add multi-tenant isolation tests.
// Background · from #Open models
Full timeline →- Nov 2 NVIDIA introduces NemoClaw open blueprints: OpenClaw, Hermes Agent or LangChain Deep Agents paired with Nemotron models and the OpenShell runtime
- Nov 2 NVIDIA builds a fully local voice agent on Nemotron models: a median 1.70 seconds from end of speech to audio, 0.72 seconds for cached replies
- Nov 1 NVIDIA runs Nemotron 3.5 Lightning on DGX Spark in a few commands with TensorRT Edge-LLM, and DSpark speculative decoding lifts HumanEval generation from 73.4 to 126.9 tokens/s