Tools #Coding agents Nov 14, 2026 · Originally published Nov 13

AWS shows how to route coding agent requests with NVIDIA NeMo Switchyard, judged by cost per solved task

Coding agents cost by the task, not the call. The two strategies in the open-source Switchyard router, and a sample for running the comparison on Bedrock.

Image: AWS Machine Learning Blog
Primary source AWS Machine Learning Blog · aws.amazon.com Read the original ↗

// Key points

  • Routing is judged by cost per solved task: total spend across runs divided by the runs whose tests pass, since failed and unfinished attempts are billed too.
  • Switchyard's LiteLLM integration has two strategies: stage reads recent tool activity and results in the request history and picks one of two candidate models, so the agent must send full history; random picks by weight for traffic splits and A/B comparisons. Neither calls a judge model.
  • The sample uses AWS CDK to deploy LiteLLM with the Switchyard plugin on ECS Fargate in private subnets behind an ALB and WAF; the switchyard-server 0.3.0 it installs is documented by NVIDIA as a demo and evaluation component, not for production.

Builder's takeSame direction as Hydrogen's Orchestrator two days ago: switch models by task difficulty. Switchyard is still a demo component and I wouldn't put it in production, but its evaluation method is worth adopting now: take 20 of your real tasks and measure cost per solved task for single models and for a routed group, which tells you far more than price per million tokens.

// Background · from #Coding agents

Full timeline →
  1. Nov 14 GitHub Copilot retires six models on December 14, including Claude Fable 5, Gemini 4 Argon and Grok 4.6, each with a suggested replacement
  2. Nov 13 Google releases Gemini 4 Hydrogen at $1.50 input and $7.50 output per million tokens, scoring 51 on the Artificial Analysis Intelligence Index, just above Argon's 50
  3. Nov 13 Google pairs Gemini 4 Argon with the Z3 constraint solver to find Go bounds panics: all three planted bugs found plus three latent ones, two of which fuzzing missed

Every Monday, the week in AI in one email

Every Monday morning: the 6 AI stories that mattered most in the past seven days, with my take, in your inbox.

Your email is only used to send the weekly and is never shared. Every email has a one-click unsubscribe link. See the latest issue →