Nov 14, 2026 · Sat · 10 items

llama.cpp reaches 1.0, ChatGPT Wallet lets agents pay, AI pentest frameworks up 30x in a month

Today's thread is where agents' boundaries sit: they can now spend money (ChatGPT Wallet) and take on a user's permissions (Cognito token exchange), while attackers use the same kind of agents to hunt for holes at scale. If you build products, approvals, permissions and spending caps need to ship before the features.

RSS
    • libllama (llama.cpp) and libggml (ggml) both ship v1.0.0; from now on API and ABI incompatibilities are signalled by a major-version bump.
    • Rules: a breaking change to the public C API in ./include bumps MAJOR; backward-compatible features, backend updates and new model support bump MINOR; fixes with no API change bump PATCH.
    • It supports more than 150 model architecture variants and all mainstream speculative decoding methods.
    • Distribution covers Docker, Debian, Homebrew, conda-forge, MacPorts, Winget and an XCFramework for iOS and macOS; CI runs on self-hosted hardware from NVIDIA, AMD, Qualcomm, Arm, IBM and Intel.

    Builder's takeThe point here is not a new feature but being safe to depend on: shipping libllama in a desktop or mobile app used to mean re-checking the interface on every upgrade. I would pin to 1.x and let MINOR updates flow; for cases like AI Cloud Drive that want on-device file understanding, an offline path is now worth a serious look.

    • ChatGPT Wallet lets agents in ChatGPT Work, Codex and dots complete purchases.
    • Each purchase requires user approval and keeps existing card protections.

    Builder's takeOnce agents can spend money, who approves what becomes the core of product design. I would first list every action in my products that costs money and mark which need a human to confirm and which can pass automatically under a limit, then decide on integration once OpenAI says whether it opens to third parties.

    • ARTEX, an open-source autonomous pentest system, was taken down by its creator on October 8, but there are now over 12,000 repositories with several hundred forks carrying their own changes, plus hundreds of similar orchestration frameworks.
    • In the last three days over 50 framework families tried to scan Cloudflare customers from thousands of IP addresses; daily requests from AI pentest and exploitation tools have grown 30x since October 1.
    • Some of the more complex attacks chain stolen credentials to move laterally; the frameworks plug into MCP and run parallel agents to enumerate, verify and attack.
    • Cloudflare's six recommendations: a WAF with rate limits, anomaly detection on web apps, bot defense, identity-gated access for high-risk apps, enforced schema validation, and threat intelligence.

    Builder's takeSmall products used to feel too minor to target; now attack cost is close to inference cost, and small sites get scanned systematically too. I would do the two cheapest things first: put every admin backend behind an identity gate, and add rate limits and schema validation to public APIs. Rotate credentials early as well, since these frameworks use leaked ones to move inward.

    • From December 14 these leave Copilot Chat, inline edits, ask and agent modes and code completions: Claude Fable 5 (replace with Fable 5.2), Claude Haiku 4.5 (Haiku 5.5), Gemini 3.8 Flash (Gemini 4 Hydrogen), Gemini 4 Argon (Gemini 4 Carbon), Grok 4.6 (Grok 4.7) and Kimi K3 (replacement to be announced).
    • Claude Haiku 4.5 stays available to individual subscribers on annual plans.
    • Under default model enablement, Business and Enterprise get the suggested replacements turned on automatically unless an admin has turned off the global default or disabled the model.

    Builder's takeArgon only became generally available two months ago and is already being replaced by Carbon; model lifecycles are now measured in months. Hydrogen also became generally available in Copilot the same day. Don't hard-code model names in scripts or team guidelines; describe tiers instead, and run your common tasks on the replacements before 12-14. Note Carbon's price change after 11-30, so Argon to Carbon is not necessarily a like-for-like swap.

    • Kitesurf runs entirely on Cloudflare Workers: it parses HTML, applies CSS, executes page JavaScript and renders screenshots without a browser container, and supports the Chrome DevTools Protocol so clients like Puppeteer can connect.
    • It is built from open-source Rust packages compiled to WebAssembly: Blitz (DOM, layout, painting), Stylo (CSS), Boa (JavaScript, only as a last resort when Dynamic Workers can't run it), Linebender (text and 2D graphics) and Krilla (PDF output).
    • It has three parts, Engine (sessions and connections), PageScript (running the page) and PageRenderer (drawing); it runs locally or deploys to your own account, which requires a Workers Paid plan.

    Builder's takeMost agent browsing is reading content and taking a screenshot, and full Chromium is heavy for that. I would move tasks like reading pages or checking layout after publishing to Kitesurf first and keep complex interaction on Chromium; being open source also means no lock-in to one hosted service.

    • A service or agent can exchange a user's token for a new one scoped to a specific downstream service: narrower than the original, bound to a single audience, and without a refresh token.
    • With an actor token, Cognito adds an act claim identifying the intermediary, so downstream services can tell an agent's request from the user's own; the example given is an MCP server exchanging for a narrow token to call a backend API.
    • It works through the token endpoint or the new GetDerivedToken API, on the Essentials and Plus feature plans in all Regions where Cognito is available.

    Builder's takeMany MCP servers today pass the user's original token all the way down, which over-grants access and makes it hard to tell who did what. Teams on Cognito can switch to token exchange directly; others should still follow the RFC 8693 pattern: exchange for a narrow token at each hop and log the act claim.

    • Available to all pay-as-you-go customers, starting with Workers and Durable Objects with more products to follow; set a limit, a notification email and multiple alert thresholds.
    • When the limit is hit a blocking policy pauses only cost-incurring customer-code invocations, such as Workers Cron Triggers and Queue consumers, while static assets keep serving.
    • The dashboard shows current-cycle spend per product in near real time (under 10 minutes) with an audit log; it is also configurable through the cf CLI and REST API.
    • It pairs with AI Gateway's existing per-user and per-model budgets to cap both compute and AI inference costs.

    Builder's takeFor indie developers, an agent loop running away overnight and burning a month's budget is a real risk, and a hard cap beats an alert. I would cap unattended parts like cron jobs and queue consumers first, at two to three times normal spend. Like yesterday's AWS guide to capping Bedrock spend per identity, cost control is becoming a platform default.

    • GLM-5.3 NVFP4 is a quantized version of Z.ai's MoE model: 753 billion parameters with 40 billion active per token, about 464 GB, more than the 252 GB of HBM3e on one GB300.
    • Two DGX Stations linked over ConnectX-8 RDMA use tensor parallelism (TP=2), each holding about half the weights, behind one OpenAI-compatible endpoint; vLLM runs with multi-token prediction, FP8 KV cache and prefix caching.
    • On the SPEED-Bench Coding dataset, total output throughput peaks at 607.8 tokens per second at concurrency 16, then drops to 385.0 at concurrency 32 with a mean request latency of 453.5 seconds.

    Builder's takeThe useful number is that throughput falls at concurrency 32 with 453.5-second mean latency: running a frontier open model locally works, but only for small internal teams at low concurrency. If you want to self-host to keep data in-house, measure the best concurrency on your own prompts the way this post does, then compare per-token cost with on-demand cloud calls.

    • 8,115 questions: 1,557 from Stack Overflow, 184 from Stack Exchange, 102 curated by hand and 6,272 generated from WikiHow guides; built with Carnegie Mellon, NVIDIA and Johns Hopkins.
    • It evaluates Claude Sonnet 4.6, GPT-5, Gemini 3.1 Pro, Qwen3.5-9B and DeepSeek-V4 Pro. On Stack Overflow questions, only 35% to 41% of model answers focus primarily on the underlying goal, versus 47% to 57% on the stated request.
    • On the same set, solutions to the stated request appear in 75% to 86% of answers, versus 59% to 71% for the underlying goal; human reference answers steer users away from the mistaken approach more often.

    Builder's takeThis is what assistant products most often miss: a specific question is not necessarily the right one. In AI Interview a common ask is how to answer a question more impressively, when the real gap is understanding the role. I would add a step to the system prompt to first infer the user's goal and whether their approach can reach it, then regress against XYBench's programming subset.

    • Routing is judged by cost per solved task: total spend across runs divided by the runs whose tests pass, since failed and unfinished attempts are billed too.
    • Switchyard's LiteLLM integration has two strategies: stage reads recent tool activity and results in the request history and picks one of two candidate models, so the agent must send full history; random picks by weight for traffic splits and A/B comparisons. Neither calls a judge model.
    • The sample uses AWS CDK to deploy LiteLLM with the Switchyard plugin on ECS Fargate in private subnets behind an ALB and WAF; the switchyard-server 0.3.0 it installs is documented by NVIDIA as a demo and evaluation component, not for production.

    Builder's takeSame direction as Hydrogen's Orchestrator two days ago: switch models by task difficulty. Switchyard is still a demo component and I wouldn't put it in production, but its evaluation method is worth adopting now: take 20 of your real tasks and measure cost per solved task for single models and for a routed group, which tells you far more than price per million tokens.

Researched and drafted with AI assistance; editorial standards and views set by Darius.

Every Monday, the week in AI in one email

Every Monday morning: the 6 AI stories that mattered most in the past seven days, with my take, in your inbox.

Your email is only used to send the weekly and is never shared. Every email has a one-click unsubscribe link. See the latest issue →