SageMaker AI training and processing jobs support lazy loading of SOCI-indexed images: a 13.2 GB image downloads in about 6 seconds instead of 3 min 42 s
A ready-made fix for GPUs idling while large images pull, with no change to job definitions.
// Key points
- SOCI indexing fetches only the files needed to start the container and streams the rest in the background; job definitions, entry points and data channels are unchanged, with a fallback to a full download if there is no index or lazy loading fails.
- Measured image download times: 1.8 GB from 48 s to 3–11 s, 13.2 GB from 3 min 42 s to about 6 s, 35.4 GB from 7 min 17 s to 3–11 s; GPU idle time during image pulls dropped by up to 90%.
- The first epoch takes 3–16 s longer, after which time per epoch matches the standard image; each tested job finished 1–6 minutes sooner overall.
- It is available in all AWS Regions for Amazon ECR private repositories only; images under about 4 GB see small gains, and the runtime is containerd, so code relying on the Docker CLI or socket may need changes.
Builder's takeSaving a few minutes per job looks minor, but for frequent short jobs such as evaluations, small fine-tunes and data processing, the GPU-hour bill drops noticeably. Before switching, check whether your training scripts call docker; if they do, that is the only thing to change. Images under 4 GB are not worth the effort.