PyTorch 2.15 on Mac: MoE models decode 5–6.5x faster, Qwen3.6-35B-A3B rising from 9.4 to 61.3 tokens/s on an M3 Ultra
Hugging Face's PyTorch team replaced many MPS bottlenecks with dedicated Metal kernels, making local models on a Mac more practical.
// Key points
- MoE models decode 5–6.5x faster than in 2.14: on an M3 Ultra with torch.compile, Qwen3.6-35B-A3B goes from 9.4 to 61.3 tokens/s and LFM2.5-8B-A1B from 23.6 to 123.6 tokens/s.
- FP8 comes to MPS, storing float8_e4m3fn in half the space of BF16; softmax over the vocabulary is about 8x faster.
- Wan2.2's video VAE decode now peaks at 8 GiB of memory instead of 270 GiB.
- MPS now requires macOS 15 or later, and PyTorch 2.15 requires Python 3.11 or later.
Builder's takeA 35B MoE at 61 tokens/s is enough for local development and offline batch work. I'll move some internal evaluation scripts to a Mac with an MoE model for a first pass and skip spinning up GPU machines for small experiments; before upgrading, check you're on macOS 15 and Python 3.11 or later.
// Background · from #Open models
Full timeline →- Oct 21 TorchTPU makes TPU a native PyTorch device; Transformers, Diffusers and Kernels run by changing the device name
- Oct 20 AWS contributes bi-directional KV transfer to vLLM, cutting time to first token in multi-turn agent sessions by up to 3.3x
- Oct 20 Hugging Face and AWS launch TorchNeuron: Transformers and friends run directly on Trainium, with 89% of the top 100 text-generation architectures working out of the box, up from 46%