AWS distills a GLM-5.3 shopping agent into Qwen3.5-4B: held-out success rises from 0.297 to 0.541, level with the teacher
A full sequence-level distillation recipe for a multi-step agent, with numbers showing a 4B student can take over a narrow, repetitive agent task.
// Key points
- GLM-5.3 on Bedrock generated 850 trajectories on ShoppingBench's 426 training problems, keeping only the solved ones; Qwen3.5-4B was fine-tuned with LoRA (rank 64) through serverless SFT on SageMaker AI
- Average success on 324 held-out problems: base model 0.297, distilled student 0.541, teacher 0.540 greedy and 0.511 sampled, a medium-sized frontier model 0.546
- General instruction following drops: IFEval prompt-strict falls from 0.819 to 0.741 and instruction-strict from 0.872 to 0.813
- Keeping the reasoning alongside the actions is what carried the student most of the way; it serves on a single G-family instance (A10G/L40/L40S), and AWS publishes no per-task cost
Builder's takeThis is the clearest recipe I've seen for moving a fixed-format agent task off a frontier API: log trajectories, keep only the successful ones with their reasoning, and fine-tune a 4B model. I'd try it first on PandaClaws' highest-volume background steps, keep a general-ability regression check like IFEval, and only switch when traffic is steady enough that an hourly instance beats per-token billing.
// Background · from #AI agents
Full timeline →- Nov 3 Google DeepMind's PlanAssist moves into a 12-month beta with UK councils after pilots cut officer assessment time by up to 58%
- Nov 3 OpenAI introduces LifeVal, a benchmark of how well AI helps people complete everyday tasks
- Nov 2 NVIDIA introduces NemoClaw open blueprints: OpenClaw, Hermes Agent or LangChain Deep Agents paired with Nemotron models and the OpenShell runtime