Microsoft Research discloses Lazarus: reprogramming fewer than 10 dormant neurons, with no training, backdoors an agent built on the model
A backdoor written straight into the weights: the model still passes standard evals, and only when a trigger appears does the agent run an attacker's command. How it works, why it is hard to catch, and the defenses Microsoft points to.
// Key points
- The researchers find that reprogramming fewer than 10 rarely firing neurons is enough to backdoor a model, with no training, needing only inference-level compute and write access to a checkpoint.
- The trigger can be mundane, such as the clock passing midnight into 2027: mid-task, the agent then emits a command that downloads and runs the attacker's code, with whatever access it was given.
- Frontier agents can spot anomalous weight edits when asked to look, but no scan can prove a model is safe; reliable detection requires diffing against the original weights, so provenance and integrity are what matter. The author stresses this is not a case against open weights: internal models are equally exposed.
- Microsoft says Foundry ties each onboarded open-weight model to its original publisher and runs it in separately scanned containers, and it is building agent-based weight scanning, red-teaming tools and cryptographic model signing with verifiable provenance.
Builder's takeTwo things I would do right away: in the deploy pipeline, check self-hosted open-model weights against the publisher's hashes and give only the release account write access to the bucket that holds them; and default agent environments to an egress allowlist, so even a tampered model cannot complete the download-and-run step.
// Background · from #Open models
Full timeline →- Nov 12 Tether AI Research releases BotPsy, a 0.8B on-device agent model: BFCL v4 rises from 23.4 to 44.0, and it beats Qwen3.5-2B-Think, 2.5x its size, on five of six tool-use benchmarks
- Nov 11 NVIDIA open-sources Snapshot: inference replicas restore from a GPU memory checkpoint in 3.6 to 42.7 seconds, 8x to 36x faster than a cold start
- Nov 11 Hugging Face defines a common interface for decision models: Transformers adds a DecisionPipeline, and llama.cpp supports them too