Tools #Open models Nov 19, 2026 · Originally published Nov 18

AWS Lambda Managed Instances adds GPUs: run inference functions on EC2 G-series instances, with vLLM, SGLang and PyTorch base images

GPU inference becomes an event-driven step without managing drivers, frameworks or scaling. For teams with bursty inference work that do not want to run a fleet of GPU instances.

Primary source AWS · aws.amazon.com Read the original ↗

// Key points

  • GPU-accelerated Event Functions run on EC2 G-series instances, scaling on GPU utilization, CPU utilization and concurrency, with multiple invocations per execution environment.
  • GPU base images come with vLLM, SGLang and PyTorch; container images and NVMe ephemeral storage go up to 32 GB each, with SOCI lazy loading and streaming from S3.
  • Pricing: the EC2 On-Demand price (Savings Plans and Reserved Instances apply), plus a 9.75% Lambda management fee and $0.20 per million requests.
  • It is available in all commercial regions where both Lambda Managed Instances and G-series instances exist.

Builder's takeIn a content backend like PandaClaws, transcription, frame extraction and small-model scoring come in bursts, and a dedicated GPU sits idle most of the time. This Lambda shape lets "upload triggers GPU processing" plug straight into event sources. I would move one self-hosted small model, such as open-source speech transcription, first, and weigh the 9.75% fee against idle GPU cost before going further.

// Background · from #Open models

Full timeline →
  1. Nov 19 Cloudflare opens the weights of Clairic and Clairic-flash: 27B and 9B extraction models that read PDFs and bill only for input tokens
  2. Nov 19 Hugging Face shows Transformers streaming MoE experts from SSD: a 180B-parameter model runs at about 14 tok/s on a 32 GB MacBook
  3. Nov 19 Rumble-2-Flash is open-sourced under Apache 2.0: a 0.5B-parameter speech recognition model for 40+ languages, averaging 6.96% error on the multilingual track

Every Monday, the week in AI in one email

Every Monday morning: the 6 AI stories that mattered most in the past seven days, with my take, in your inbox.

Your email is only used to send the weekly and is never shared. Every email has a one-click unsubscribe link. See the latest issue →