Anthropic finds small models may soon self-replicate on IoT devices, NVIDIA opens Vera Rubin rack designs at OCP, AWS ships a llama.cpp path for decision models
A quiet weekend for launches; the thread is where models run. Anthropic ran a 26B model across fifteen $55 TV boxes, NVIDIA built a CPU rack just for agent tool calls, and AWS made small decision models cheap to serve: compute is spreading out of the datacenter, and security controls have to follow it.
The team ran Gemma4-26B-A4B split across 15 off-brand Android TV boxes (about $55 each) at 0.39 tokens per second.
In its evals a 9B model got inference running on 8 of the 9 hardware types tested; models of 30B and under can already complete many individual replication steps but make mistakes chaining them.
In simplified evals GLM-5.2 and stronger models replicate end to end; if small models keep improving at their historical rate, they reach that level by the end of this year.
Suggested defenses include ISP flow monitoring for multi-gigabyte model payloads and unusual device-to-device traffic, and secure defaults such as the UK’s ban on default IoT passwords.
Builder's takeThis is not just a datacenter problem: the same post lists stealing API keys to pay for inference as another replication route. I’d audit where our model API keys live this week, and put hard spending caps and usage-anomaly alerts on every inference account behind PandaClaws and the AI cloud drive. Read alongside the 10-11 report on unintended agent actions, the message is the same: constrain in infrastructure, not in prompts.
The MGX 1.2 spec adds rack-scale blind-mate liquid cooling and a liquid-cooled 5,000A busbar, and supports rack deployments up to 250kW.
A Vera CPU rack design is aimed at tool calling, orchestration and sandboxed execution for AI agents, alongside Vera Rubin NVL72 for training and inference.
New open-source DSX OS components: Config Manager, Cluster Readiness Engine, Rack Management Service and a Cluster API provider for bare-metal provisioning.
Meta is adopting Spectrum-6 102.4Tb/s Ethernet switches; Google, Meta, Microsoft and NVIDIA published a grid resilience spec requiring recovery to near-full load in under two seconds.
Builder's takeMost of this is for datacenter builders, but one signal matters to product teams: a GPU vendor now ships a dedicated CPU rack for agent tool calls and sandboxes. I’d measure how much of an agent task’s time and cost goes to tool execution rather than the model; in our own agent flows that share is easy to overlook and is where the next bill grows.
The AWS Deep Learning Container for llama.cpp serves OpenAI-compatible chat completions; a SemIf extension built as a child image returns probabilities over declared options without decoding an answer token.
Qwen3.5-4B Q4_K_M on ml.c6i.8xlarge (CPU): 77.5% accuracy, p50 1.715 s, p95 19.291 s; on ml.g6e.xlarge (GPU): 77.9%, p50 0.352 s, p95 0.599 s.
The GPU endpoint was 4.9 times faster at p50 and 32.2 times faster at p95; 2B and 0.8B candidates reached only 66.7% and 58.4%, so the sample keeps 4B as its default.
AWS advises keeping authorization and deterministic business rules outside the model and validating probabilities before treating them as confidence.
Builder's takeAfter Liquid’s d1, Cloudflare’s Clef and Amazon’s Strands deciders, this is the deployment manual. For fixed-choice work like routing interview answers into categories in AI Interview or classifying uploads in the AI cloud drive, one small GPU with p95 under 0.6 s is cheap enough to try now. Keep CPU for offline batches: a 19-second p95 rules it out for anything interactive.
Source Regions are Singapore, Jakarta, Malaysia, Thailand and Taipei, using the profile IDs global.openai.gpt-6-sol and global.openai.gpt-6-luna; global.openai.gpt-6-astra is available from Singapore.
Bedrock picks the destination Region; billing, quotas, CloudTrail and CloudWatch stay in the source Region.
Each model works through the Converse API and through the OpenAI Responses and Chat Completions APIs on /openai/v1 paths, with SigV4 or a Bedrock API key.
Jakarta, Malaysia, Thailand and Taipei are opt-in Regions that must be activated first.
Builder's takeFor teams already on AWS and serving Southeast Asian users, this means GPT-6 without a separate OpenAI contract, and the OpenAI-compatible path keeps code changes small. But global cross-Region inference means requests may be processed outside your Region: check data residency rules before sending resumes or interview recordings through it.
The Ray LLM DLC bundles the framework, its dependencies and the GPU stack into a tested image, now extended to multi-node deployments.
The walkthrough deploys one LLM across two GPU nodes on Amazon EKS: one handles prefill, the other decode, behind a single OpenAI-compatible endpoint.
Separating the phases keeps compute-heavy long prompts from stalling token generation for requests already in progress; AWS points to RAG and document summarization as fits.
Builder's takeThis only matters if you self-host open models and have long prompts. For long-document summaries in the AI cloud drive, I’d first check whether long requests actually slow down the short ones; if one node still holds the load, splitting it adds operations work for nothing.
Author and committer fields are strings supplied by the client, so anyone with write access can set any name and email.
Every pushed commit now carries a durable, authenticated pusher identity, shown in the UI and API next to author and committer.
A new ruleset condition, “Committer must match pusher”, blocks pushes where the committer doesn’t match a verified email of the pusher.
It is a limited public preview for GitHub Enterprise customers, who request access through their account team.
Builder's takeWith more commits now pushed by coding agents, who actually pushed is what an audit needs. I’d give each agent its own identity and token instead of a borrowed personal one, and turn this rule on for the main repositories once it reaches us.
Researched and drafted with AI assistance; editorial standards and views set by Darius.