Thinking Machines releases Inkling 2 Small under Apache 2.0: 594B total and 17B active parameters, text and image input, 1M-token context
Another near-frontier open-weights MoE model. Size, hardware needs and the vendor's evaluations, for teams choosing an open model for agents or document understanding.
// Key points
- A 48-layer sparse MoE decoder routing each token to 8 of 384 experts; natively multimodal with text and image input and a context window of up to 1M tokens.
- Pretraining, midtraining and post-training were all done in NVFP4; the NVFP4 checkpoint needs at least 500 GB of VRAM (e.g. 2x B300 or 4x H200), the MXFP8 one at least 800 GB.
- Local inference is supported on SGLang, vLLM, TokenSpeed and Unsloth; API access is available through NVIDIA, Fireworks, Together, Baseten, Nebius, Modal, Cloudflare and others.
- In the vendor's table it scores 30.6 on the Artificial Analysis Index v5.0 (GLM-5.3-Flash 27.8, DeepSeek-V4.1-Flash-1017 32.6), 90.2% on BrowseComp and 94.6% on OmniDocBench v1.6.
Builder's takeIt says Small, but this is not a model you run locally: the floor is two B300s. I would go through a hosted API first and compare cost per task against DeepSeek V4.1-Flash on my own work; the OmniDocBench score makes me want to try it on scanned documents and tables in AI Cloud Drive, but every number here is self-reported, so retest on your own samples.
// Background · from #Open models
Full timeline →- Nov 14 llama.cpp and ggml reach v1.0.0: breaking changes to the public C API will now bump the major version
- Nov 14 NVIDIA serves the 753B-parameter GLM-5.3 across two DGX Stations, peaking at 607.8 output tokens per second at concurrency 16
- Nov 13 IBM releases the LLMON markup language with Granite 4.1 3B and 8B preview models: reserved special tokens separate instructions from data, blocking 47% more injection attacks on OPI than the best frontier model tested