Hugging Face defines a common interface for decision models: Transformers adds a DecisionPipeline, and llama.cpp supports them too
Picking from fixed options, as in classification and routing, is slow and fragile when a generative model has to emit JSON. Decision models return a probability for each option instead.
// Key points
- A decision model's interface is state plus questions plus allowed answers, returning decisions plus probabilities; instead of generating token by token, it scores answers the application already understands.
- It defines three primitives: Choice (a probability distribution over options), Score (a rating on an ordered scale) and Noul (the probability of a yes/no answer).
- Transformers adds DecisionPipeline, AutoModelForDecision and DecisionConfig, with fine-tuning through the Trainer API; llama.cpp can serve GGUF decision models through the same interface, and the Hub has a new decision-model tag.
- The goal is a shared interface without a shared implementation: models can be built on encoders such as ModernBERT or on larger generative backbones.
Builder's takeOur products make many calls like "who should handle this request" or "which role does this resume fit", mostly by asking a large model for JSON and parsing it. Resume screening in Smart Interview and file classification in AI Cloud Drive are good first tests for a decision model: check accuracy holds, then measure latency and cost per call, and use the probabilities directly as thresholds for human review.
// Background · from #Open models
Full timeline →- Nov 11 NVIDIA open-sources Snapshot: inference replicas restore from a GPU memory checkpoint in 3.6 to 42.7 seconds, 8x to 36x faster than a cold start
- Nov 10 NVIDIA Nemotron 3.5 Super VL arrives in SageMaker JumpStart: 120B total and 12B active parameters, a 1M-token context, and an NVFP4 variant of about 75 GB
- Nov 10 Amazon's HOPE prunes MoE experts by modeling how they work in pairs, ranking 1.58 on average at 50% pruning versus 2.42 for the next-best method, REAP