Open source #Open models Oct 20, 2026 · Originally published Oct 19

Hugging Face and AWS launch TorchNeuron: Transformers and friends run directly on Trainium, with 89% of the top 100 text-generation architectures working out of the box, up from 46%

Trainium goes from needing model-specific code to changing a device name, a key step in competing with GPUs for open-model workloads.

Primary source Hugging Face Blog · huggingface.co Read the original ↗

// Key points

TorchNeuron is a PyTorch plugin that exposes neuron as a native device, supporting eager mode, torch.compile and distributed APIs such as FSDP and tensor parallelism.
Transformers, Diffusers, Accelerate, PEFT and TRL run on Trainium directly; for inference you pass device_map="neuron", and the authors recommend these libraries over Optimum Neuron.
89% of the 100 most-used text-generation architectures work out of the box, up from 46% with Optimum Neuron v0.4.5.
It enters public beta on October 20; install with pip install torch-neuronx.

Builder's takeTeams fine-tuning or self-hosting open models on AWS can put Trainium back on the shortlist: the blocker used to be rewriting model code, and now it's changing a device. I'd take one inference service already working on GPUs, move it over unchanged and compare cost per million tokens; it's still a public beta, so don't move training first.

// Background · from #Open models

Full timeline →