AWS Lambda Managed Instances adds GPUs: run inference functions on EC2 G-series instances, with vLLM, SGLang and PyTorch base images
GPU inference becomes an event-driven step without managing drivers, frameworks or scaling. For teams with bursty inference work that do not want to run a fleet of GPU instances.
// Key points
- GPU-accelerated Event Functions run on EC2 G-series instances, scaling on GPU utilization, CPU utilization and concurrency, with multiple invocations per execution environment.
- GPU base images come with vLLM, SGLang and PyTorch; container images and NVMe ephemeral storage go up to 32 GB each, with SOCI lazy loading and streaming from S3.
- Pricing: the EC2 On-Demand price (Savings Plans and Reserved Instances apply), plus a 9.75% Lambda management fee and $0.20 per million requests.
- It is available in all commercial regions where both Lambda Managed Instances and G-series instances exist.
Builder's takeIn a content backend like PandaClaws, transcription, frame extraction and small-model scoring come in bursts, and a dedicated GPU sits idle most of the time. This Lambda shape lets "upload triggers GPU processing" plug straight into event sources. I would move one self-hosted small model, such as open-source speech transcription, first, and weigh the 9.75% fee against idle GPU cost before going further.
// Background · from #Open models
Full timeline →- Nov 19 Cloudflare opens the weights of Clairic and Clairic-flash: 27B and 9B extraction models that read PDFs and bill only for input tokens
- Nov 19 Hugging Face shows Transformers streaming MoE experts from SSD: a 180B-parameter model runs at about 14 tok/s on a 32 GB MacBook
- Nov 19 Rumble-2-Flash is open-sourced under Apache 2.0: a 0.5B-parameter speech recognition model for 40+ languages, averaging 6.96% error on the multilingual track