Open source #Open models Nov 11, 2026 · Originally published Nov 10

NVIDIA open-sources Snapshot: inference replicas restore from a GPU memory checkpoint in 3.6 to 42.7 seconds, 8x to 36x faster than a cold start

When scaling up or recovering, a model cold start often takes minutes. How Snapshot skips weight loading, compilation and graph capture.

Image: NVIDIA Technical Blog
Primary source NVIDIA Technical Blog · developer.nvidia.com Read the original ↗

// Key points

  • Snapshot is an open-source checkpoint and restore system for GPU workloads on Kubernetes; it captures a fully initialized workload's CPU and GPU memory and restores it on nodes with the same GPU model and driver.
  • In tested configurations restores take 3.6 to 42.7 seconds, 8x to 36x faster than cold starts that take minutes; creating a checkpoint takes 15 seconds to 5.5 minutes, and checkpoint plus restore is shorter than a single cold start in every configuration.
  • It supports vLLM, SGLang and TensorRT-LLM, plus multi-GPU setups with tensor, data and expert parallelism, with checkpoints up to 1 TB across eight GPUs for large MoE models.
  • One upstream contributor cut GLM-5.3 Pod readiness from about 8 minutes to 31 seconds on NVIDIA HGX B300.

Builder's takeFor teams self-hosting open models, the biggest cost is often the GPUs kept warm for peaks. With cold starts down from minutes to under a minute, scaling idle replicas to zero becomes realistic. I would start with batch-style services that tolerate tens of seconds of delay, and check whether the same-GPU-model-and-driver constraint fits our fleet.

// Background · from #Open models

Full timeline →
  1. Nov 11 Hugging Face defines a common interface for decision models: Transformers adds a DecisionPipeline, and llama.cpp supports them too
  2. Nov 10 NVIDIA Nemotron 3.5 Super VL arrives in SageMaker JumpStart: 120B total and 12B active parameters, a 1M-token context, and an NVFP4 variant of about 75 GB
  3. Nov 10 Amazon's HOPE prunes MoE experts by modeling how they work in pairs, ranking 1.58 on average at 50% pruning versus 2.42 for the next-best method, REAP

Every Monday, the week in AI in one email

Every Monday morning: the 6 AI stories that mattered most in the past seven days, with my take, in your inbox.

Your email is only used to send the weekly and is never shared. Every email has a one-click unsubscribe link. See the latest issue →