Open models
Open weights, licenses and small models you can run locally. Every brief item about Open models, newest first, each with its primary source.
Timeline 92 items
-
Oct 21 · Wed · 2 items
- Open source huggingface.co ↗TorchTPU makes TPU a native PyTorch device; Transformers, Diffusers and Kernels run by changing the device name
Take · Read with yesterday's TorchNeuron, PyTorch has taken both Trainium and TPU this week. For product teams, inference pricing is no longer GPU quotes alone; my advice is still to move one running inference service over unchanged and compare unit cost, rather than rewrite code for new hardware.
From the Oct 21 brief · item 08 →
-
-
Oct 20 · Tue · 2 items
- Open source aws.amazon.com ↗AWS contributes bi-directional KV transfer to vLLM, cutting time to first token in multi-turn agent sessions by up to 3.3x
Take · If you self-host inference for agents, this is one of the most practical latency fixes this year: agent sessions are long-history and multi-turn by nature, and recomputing history is the main reason first tokens are slow. It assumes you already split prefill and decode; if you haven't, don't re-architect for it, pin sessions to one node and turn on prefix caching first, which is cheaper.
From the Oct 20 brief · item 03 → - Open source huggingface.co ↗Hugging Face and AWS launch TorchNeuron: Transformers and friends run directly on Trainium, with 89% of the top 100 text-generation architectures working out of the box, up from 46%
Take · Teams fine-tuning or self-hosting open models on AWS can put Trainium back on the shortlist: the blocker used to be rewriting model code, and now it's changing a device. I'd take one inference service already working on GPUs, move it over unchanged and compare cost per million tokens; it's still a public beta, so don't move training first.
From the Oct 20 brief · item 04 →
-
-
Oct 19 · Mon · 1 item
- Industry blogs.nvidia.com ↗NVIDIA RTX Spark laptops go on sale from $2,399 with up to 128GB of unified memory
Take · 128GB of unified memory leaves plenty of headroom for models like Nemotron 3.5 Bolt, released on 10-15 and fitting in 24GB. Don't refresh everyone's laptop yet: give one to whoever evaluates models, run a week of offline tests on private data (documents in AI Cloud Drive that shouldn't go to the cloud, say), then compare against pay-per-use APIs.
From the Oct 19 brief · item 06 →
-
-
Oct 18 · Sun · 4 items
- Model release huggingface.co ↗DeepSeek releases V4.1-Flash-1017 under MIT: DeepSWE v1.1 rises from 74.2 to 77.0 and Terminal-Bench 4.0 from 31.2 to 42.3
Take · Put it next to Beam on 10-17 and Muse Spark Open on 10-16 and open coding models closed in on closed ones all week. I'd point Claude Code at it on a batch of real tasks from my own repos, checking whether shorter outputs still mean complete changes; note the missing Jinja template before self-hosting, and don't force a generic one onto it.
From the Oct 18 brief · item 01 → - Open source huggingface.co ↗Tencent Hunyuan open-sources Hy-Memory-30B-A3B under Apache 2.0: a roughly 3B-active memory writer averaging 55.49 across four memory frameworks, ahead of Claude Opus 5 at 54.62
Take · Memory writing is a high-frequency background call, and running it on a flagship model is the most expensive and least sensible choice. I'd swap it in on Mem0 for a week for the user-preference memory in AI Cloud Drive; as my 10-17 post on aipan's cost bill notes, long-run cost piles up in repeated background steps like this. The scores come from 20% subsamples, so check on your own data first.
From the Oct 18 brief · item 03 → - Open source huggingface.co ↗NVIDIA releases Dory1.0, a looped 4B model that trades compute for accuracy by loop count: SWE-bench Verified goes from 50.3 at one loop to 61.8 at four
Take · One model with compute set per request suits on-device and small teams: one loop for easy requests, four for hard ones, without maintaining two models. I'd measure the real loop-versus-latency curve in an offline batch job first; it's English-only, so Chinese products shouldn't ship it yet.
From the Oct 18 brief · item 04 → - Open source huggingface.co ↗NVIDIA and AI4Bharat open a 2.8 trillion-token Indic pretraining corpus covering 22 languages, along with the pipeline that generated it
Take · For us the pipeline matters more than the Indic data itself: translate whole documents, score them, repair only the low scorers and preserve code and equations. That's a recipe I could borrow to localize the AI Interview question bank. Run it on 100 questions first and spot-check how much the repair step actually fixes.
From the Oct 18 brief · item 05 →
-
-
Oct 17 · Sat · 3 items
- Open source huggingface.co ↗Reflection opens Beam under Apache 2.0: a 501B-total, 23B-active MoE for coding and agents with a 1M context
Take · Compared with Muse Spark Open on 10-16, Beam’s big difference is a stated Apache 2.0 license, which makes it the first frontier-class open coding model I’d put on a self-hosting shortlist. Four B200s means it’s only for teams with the hardware; I’d first run it through an inference provider on real tasks from my own repos, compare per-task cost with closed models, then decide whether to host it.
From the Oct 17 brief · item 01 → - Open source huggingface.co ↗IBM releases Granite Switch 4.1 in 8B and 30B: chat, decisions and speech-to-text from one endpoint, using about the base model’s VRAM
Take · Decision models keep piling up this week, and Granite’s approach is the cheapest: instead of a separate model per classifier, one base with many adapters. For content moderation and file classification in AI Cloud Drive I’d try the 8B first and measure how much latency drops when one upload triggers several decisions over a shared KV cache.
From the Oct 17 brief · item 04 → - Open source huggingface.co ↗Black Forest Labs opens FLUX.3 Image Klein: text-to-image, multi-reference editing and bounding-box layout in one model, about 2 seconds per 1024 image on an H100
Take · PandaClaws generates a header image for every post, and the pain is that text and subjects land anywhere; bounding-box layout control hits exactly that. I’d run the distilled checkpoint on a batch of real posts and compare per-image cost and redo rate; the post doesn’t state a license, so check the model page before commercial use.
From the Oct 17 brief · item 05 →
-
-
Oct 16 · Fri · 3 items
- Open source huggingface.co ↗Meta releases open weights for Muse Spark Open, an 816B-total, 42B-active multimodal MoE with a 1M context
Take · The Hugging Face post calls it the most capable US open-weights model to date, but for a small team like mine it is a frontier model you can self-host rather than one you casually run: a single B200 node is the entry point. I’ll wait for a clear license and independent evals before comparing it with closed APIs on AI Cloud Drive’s long-document Q&A; and since reasoning can’t be turned off and defaults to xhigh, work out latency and token cost first.
From the Oct 16 brief · item 01 → - Open source blogs.nvidia.com ↗NVIDIA opens the 120B Nemotron 3.5 Super VL with datasets and training recipes; the 74B-A9B Bolt fits in 24GB with native 2-bit quantization
Take · In the 10-14 issue this was still a preview; today Bolt is actually on Hugging Face, and a 74B agent model runs in 24GB. I’ll test Bolt locally on PandaClaws’ batch rewriting to see how much 2-bit quantization costs on long Chinese text; Super VL, with its datasets and recipes, suits teams that want to fine-tune. Until the license is clear, evaluate only and keep it out of production.
From the Oct 16 brief · item 04 → - Open source mistral.ai ↗Mistral releases Timestral time-series models from 5M to 209M parameters, forecasting a day ahead in 140 ms on a CPU
Take · Capacity planning and usage forecasting are things we’ve handled with rules or not at all. A zero-shot model that runs in about a hundred milliseconds on a CPU is worth a try: I’ll backtest it on AI Cloud Drive’s storage growth and model call volume from the past few months to see if it can replace our guesswork scaling thresholds. Read what the Modified MIT license changes before going to production.
From the Oct 16 brief · item 10 →
-
-
Oct 15 · Thu · 2 items
- Open source huggingface.co ↗Microsoft releases UserRepGen-4B under MIT: it turns multi-turn conversations into user descriptions for simulators, scoring 4.95 on fidelity against GPT-5.5’s 3.90
Take · The hardest thing to test in the AI interview product is the variety of candidates, and this model turns real conversations into reusable simulated users. I’d try it on de-identified interview transcripts to build a set of candidate personas for regression tests; but its re-identification ability means the descriptions are sensitive data themselves, so they stay in an internal test environment and never get stored next to resumes.
From the Oct 15 brief · item 07 → - Research huggingface.co ↗Ai2 releases HiRO-ACEv2, generating global 3 km extreme-weather simulations at about 1/300th of the cost
Take · I don’t work on weather, but the shape is worth borrowing: a cheap coarse model plus a dedicated refinement model, paying for detail only where it’s needed. Read alongside this week’s small-model releases, cheap coverage first and a specialist model for precision second is becoming a general way to cut cost.
From the Oct 15 brief · item 08 →
-
-
Oct 14 · Wed · 2 items
- Open source huggingface.co ↗Liquid AI releases open-weight LFM2.5-24B-A2B, a reasoning agent model with 2B active parameters that decodes 184 tokens/s on an M5 Max
Take · Read the table, not the headline: it wins on tool calling and instruction following and loses clearly on coding and web tasks. That makes it a candidate for the local tool-routing and form-filling steps in PandaClaws, not for a coding agent. Given the 10-13 self-replication research, if you self-host it, keep the endpoint off the public internet.
From the Oct 14 brief · item 04 → - Industry blogs.nvidia.com ↗NVIDIA lines up software for RTX Spark laptops shipping October 16, including the Nemotron 3.5 Bolt 74B agent model on October 15
Take · Together with LFM2.5 today, the local agent stack is filling in: hardware, small models, harnesses and llama.cpp support for decision models. I’d plan for “hybrid local and cloud routing” as a product feature, starting with keeping file parsing in the AI cloud drive on the user’s machine when it can, and only sending the hard questions out.
From the Oct 14 brief · item 09 →
-
-
Oct 13 · Tue · 2 items
- Safety anthropic.com ↗Anthropic’s Frontier Red Team says small models may be months from self-replicating on IoT devices; a 9B model set up inference on 8 of 9 hardware types
Take · This is not just a datacenter problem: the same post lists stealing API keys to pay for inference as another replication route. I’d audit where our model API keys live this week, and put hard spending caps and usage-anomaly alerts on every inference account behind PandaClaws and the AI cloud drive. Read alongside the 10-11 report on unintended agent actions, the message is the same: constrain in infrastructure, not in prompts.
From the Oct 13 brief · item 01 → - Tools aws.amazon.com ↗AWS shows decision models served with llama.cpp on SageMaker: a quantized 4B model scores 77.9% on 231 JevBench tasks at 0.352 s p50 on GPU
Take · After Liquid’s d1, Cloudflare’s Clef and Amazon’s Strands deciders, this is the deployment manual. For fixed-choice work like routing interview answers into categories in AI Interview or classifying uploads in the AI cloud drive, one small GPU with p95 under 0.6 s is cheap enough to try now. Keep CPU for offline batches: a 19-second p95 rules it out for anything interactive.
From the Oct 13 brief · item 03 →
-