Meta releases open weights for Muse Spark Open, an 816B-total, 42B-active multimodal MoE with a 1M context
Meta has put out an open-weight version of its Muse Spark line that takes text, images, audio and video, with deployment recipes from GPU clusters down to a Mac.
// Key points
- About 816B total parameters with about 42B active per token; 256 experts per layer, 8 picked per token; up to 1M tokens of context.
- Weights come in BF16 (about 1.5 TB) and NVFP4; NVFP4 runs in Transformers on a single B200 node, while BF16 needs four 8×H200 nodes.
- Reasoning cannot be turned off; effort levels are low, medium, high and xhigh, with xhigh as the default.
- A 2-bit llama.cpp quant ran at 18 tokens/s on a 512 GB Mac M3 Ultra using about 304 GB; the post gives no benchmark scores and does not state the license.
Builder's takeThe Hugging Face post calls it the most capable US open-weights model to date, but for a small team like mine it is a frontier model you can self-host rather than one you casually run: a single B200 node is the entry point. I’ll wait for a clear license and independent evals before comparing it with closed APIs on AI Cloud Drive’s long-document Q&A; and since reasoning can’t be turned off and defaults to xhigh, work out latency and token cost first.
// Background · from #Open models
Full timeline →- Oct 16 NVIDIA opens the 120B Nemotron 3.5 Super VL with datasets and training recipes; the 74B-A9B Bolt fits in 24GB with native 2-bit quantization
- Oct 16 Mistral releases Timestral time-series models from 5M to 209M parameters, forecasting a day ahead in 140 ms on a CPU
- Oct 15 Microsoft releases UserRepGen-4B under MIT: it turns multi-turn conversations into user descriptions for simulators, scoring 4.95 on fidelity against GPT-5.5’s 3.90