Ai2 opens ToyBlocks, a robot learning framework: 85,336 long-horizon demonstrations, about 1,600 hours of data, and up to 24x real-time generation on one GPU
A controlled tabletop world of blocks for generating tasks and training and evaluating robots, which also exposes where frontier models acting as agents fall short in planning and perception.
// Key points
- Released: simulation assets, task and demonstration generators, training and evaluation code, validation sets, and two models, Molmo2-4B-tb and maniflow-tb; the data has 85,336 long-horizon demonstrations and about 144 million frames
- With Claude Opus 5.5 as the agent, 63% of search-and-spell tasks were solved or correctly declared infeasible; learned skill descriptions raised success from 24% to 63%, versus 36% for expert-written ones
- Adding 100 world-model rollouts raised a policy's success in an unseen region from 19% to 78%; ManiFlow's atomic skills scored 80.0% in simulation and 81.1% on a real robot
- The authors say action models still generalize poorly to new spatial layouts and frontier models still misread letter orientation; a larger second release is planned within weeks
Builder's takeI don't build robots, but the finding that learned skill descriptions beat expert-written ones matters to anyone writing tool descriptions for agents: let the model rewrite them from real call results, then compare on an eval set. I would try one round on PandaClaws' tool descriptions.
// Background · from #Open models
Full timeline →- Nov 8 Kimi K3.1 Pro (Highspeed) arrives on Amazon Bedrock: open-weight, natively multimodal, a 1-million-token context, and explicit prompt caching
- Nov 8 NVIDIA open-sources the Reshard collective, making RL post-training weight refit 2.7x to 5.7x faster than gather-and-broadcast
- Nov 8 Gradium releases Phonon for on-device speech: from about 65 MB, 28 ms p95 time to first audio on CPU streaming, and an Apache 2.0 English model