Hugging Face launches a Trainium compatibility lookup: whether a Hub model runs unmodified on Trainium, with a script to run it yourself
The 10-20 issue covered TorchNeuron running Transformers directly on Trainium; this adds a lookup for which specific models actually run.
// Key points
- The new torch-neuronx model support Space lists models by Selected, Trending or Downloads, and each result shows the core count, run time including compilation, and peak device memory.
- Only Transformers text-generation and image-text-to-text models are tested for now, limited to models that fit on 4 Trainium2 devices (16 cores at most); diffusers, sentence transformers and more tasks are planned.
- Each test generates 32 tokens: single-core runs are compared with CPU output, and sharded models are checked for fluent English by a small judge model. A PASS says nothing about throughput or longer generations.
- Each model comes with its Neuron stack, packages and run.py, precompiled graphs can be synced from a public NEFF cache to skip compilation, and the underlying aws-neuron/torch-neuronx-model-support dataset can be queried with pandas.
Builder's takeIf you want to move open models to Trainium to save money, the first step is now a lookup instead of trial and error. But it only proves a model starts: I'd take the PASS models through real requests, measure throughput and long outputs, and then compare against the current GPU bill.
// Background · from #Open models
Full timeline →- Oct 25 Tencent's WeChat Vision team open-sources WeVisOne-VL 2B and 4B under Apache 2.0: one model for detection, grounding, counting, OCR, segmentation and matting
- Oct 24 AWS shows the full bill for one multi-turn RL run: about $339 took gpt-oss-20b from a 61% to an 84% Wordle solve rate
- Oct 23 SageMaker AI training sessions are generally available: write your own post-training loop and reward function while AWS runs the GPUs, billed by tokens processed