AI agents
Putting agents to work: evaluation, reliability, tool use and cost. Every brief item about AI agents, newest first, each with its primary source.
Timeline 128 items
-
Oct 22 · Thu · 2 items
- Open source huggingface.co ↗NVIDIA opens Kumo Relational, which predicts churn and fraud directly on relational databases without training and beats supervised pipelines on 41 tasks across 15 databases
Take · What small teams lack is the people to build a feature and training pipeline for every prediction task. For questions like subscription churn in AI Interview or storage growth in AI Cloud Drive, I'd run it on my own database as a baseline and compare it with rules or a simple model before investing more. The results are the vendor's own, so trust your data over them, and have someone read the weights license first.
From the Oct 22 brief · item 08 →
-
-
Oct 21 · Wed · 3 items
- Model release mistral.ai ↗Mistral releases Voxtral 3 speech models and a voice agent SDK: a 4B transcriber at $0.0035 a minute, TTS at $13 per million characters
Take · For voice-first products like AI Interview, what I care about most is end-of-turn detection and self-deployment: a bad interruption call wrecks the interview, and candidate recordings should ideally stay in my own environment. I'll first compare Chinese error rate and end-to-end latency against my current transcription on a batch of real interview recordings before deciding to switch.
From the Oct 21 brief · item 02 → - Tools aws.amazon.com ↗Amazon Bedrock adds structured outputs for all Claude models: JSON Schema output and strict tool use
Take · Teams on Bedrock can finally delete the retry-on-parse-failure glue. File metadata extraction in AI Cloud Drive is exactly this case; I'll tighten the schema to required fields only, then compare output tokens and failure rate. Constrained decoding guarantees format, not correctness, so keep sampling the results.
From the Oct 21 brief · item 06 → - Research huggingface.co ↗IBM's ALTK-Evolve appends learned rules to the prompt, taking a GPT-4.1 support agent to 67.5% Pass^4 on τ²-bench retail without touching weights
Take · This matches the approach in my checklist on fallback and human confirmation in agent systems: turning failures into explicit rules beats switching to a pricier model. I'll keep a failure log for every agent in production, distill it into rules for the prompt on a schedule, and watch every-time metrics like Pass^4, not just the average.
From the Oct 21 brief · item 09 →
-
-
Oct 20 · Tue · 2 items
- Model release blog.cloudflare.com ↗Cloudflare releases Clairic, an extraction model that returns typed values in one forward pass, billed on input tokens only, in preview on Workers AI
Take · This runs in the same direction as OpenAI's Decisions API on 10-06: pull form-filling out of generative models. Extracting dates, amounts and parties from files uploaded to AI Cloud Drive is exactly this case; I'll compare Clairic with our current LLM approach on a few hundred real files for field accuracy, and keep it out of the main path until the weights are out.
From the Oct 20 brief · item 02 → - Open source aws.amazon.com ↗AWS contributes bi-directional KV transfer to vLLM, cutting time to first token in multi-turn agent sessions by up to 3.3x
Take · If you self-host inference for agents, this is one of the most practical latency fixes this year: agent sessions are long-history and multi-turn by nature, and recomputing history is the main reason first tokens are slow. It assumes you already split prefill and decode; if you haven't, don't re-architect for it, pin sessions to one node and turn on prefix caching first, which is cheaper.
From the Oct 20 brief · item 03 →
-
-
Oct 19 · Mon · 2 items
- Industry anthropic.com ↗Anthropic publishes its own employment data: by August, interactive AI agent hours outnumbered employees six to one, and junior technical hiring shrank as a share
Take · The number to remember isn't six to one but half a work week just to read the output. I hit the same wall with PandaClaws: generation is cheap, review is scarce. Before adding another agent, build a review queue that ranks output by risk and only surfaces what needs a human; as I wrote in What Breaks First When One Person Owns Every Service, attention fails before the stack does.
From the Oct 19 brief · item 01 → - Research anthropic.com ↗Working in a closed loop with an evaluator, Claude Mythos 5 designs quantum circuits that beat the best published ones on qubits and gates in all eight tasks, one key step about 12x cheaper
Take · What interests me isn't quantum but the method: once a task can be checked automatically and given an objective score, a model can run on its own for tens of hours. In a product, pick out problems like that, such as retrieval recall in AI Cloud Drive or prompt scores on a fixed question set, write the evaluator first and let an agent iterate; it beats a human babysitting the tuning.
From the Oct 19 brief · item 04 →
-
-
Oct 18 · Sun · 3 items
- Tools api-docs.deepseek.com ↗DeepSeek opens its Search and Fetch APIs to everyone at $0.75 per 1,000 calls per endpoint, free through Claude Code and Codex until October 31
Take · Agent web access now has a plainly priced option from a Chinese provider. For a content backend that looks things up, like PandaClaws, use the free window before October 31 to compare hit rate and latency against your current search; before adopting it, count searches per task, because that number runs away easily in agents.
From the Oct 18 brief · item 02 → - Open source huggingface.co ↗Tencent Hunyuan open-sources Hy-Memory-30B-A3B under Apache 2.0: a roughly 3B-active memory writer averaging 55.49 across four memory frameworks, ahead of Claude Opus 5 at 54.62
Take · Memory writing is a high-frequency background call, and running it on a flagship model is the most expensive and least sensible choice. I'd swap it in on Mem0 for a week for the user-preference memory in AI Cloud Drive; as my 10-17 post on aipan's cost bill notes, long-run cost piles up in repeated background steps like this. The scores come from 20% subsamples, so check on your own data first.
From the Oct 18 brief · item 03 → - Safety developers.googleblog.com ↗Google OAuth adds DPoP support, binding refresh tokens to a client key pair so stolen tokens can't be replayed
Take · Products whose agents call Google Drive or Gmail for users should change how they store refresh tokens first. If AI Cloud Drive imports from Google Drive, I'd move its refresh tokens to DPoP binding with the private key kept server-side; the trade-off of binding only refresh tokens and keeping access tokens short-lived is worth copying in your own gateway too.
From the Oct 18 brief · item 07 →
-
-
Oct 17 · Sat · 2 items
- Industry blog.cloudflare.com ↗Cloudflare adds continuous checks for Verified bots: a trust band based on whether they honor robots.txt and slow down after a 429
Take · If you build browser agents or crawlers, this matters more than any model launch: getting through will depend on a behavior record, not just a signature. I’d check our crawling code reads robots.txt and backs off exponentially on a 429; that used to be courtesy, now it’s scored. Read with the Cloudflare and OpenAI discovery pilot on 10-16, Cloudflare is making itself the rule-setter for AI traffic.
From the Oct 17 brief · item 07 → - Tools aws.amazon.com ↗AWS shares the evaluation loop behind its internal sales agents: three months took the pass rate from 93% to 96% and pure hallucinations from 138 to under 52 per 10,000 responses
Take · Reporting $410.08 as $410.08 million is the scariest kind of error: confident, well formatted and hard to spot in manual checks. I’d copy the approach with a fixed question set for the dashboard Q&A in AI Cloud Drive, run on every prompt or data-pipeline change before release, and ship data, tool and prompt changes together, since the mismatch in between is where incidents happen.
From the Oct 17 brief · item 08 →
-
-
Oct 16 · Fri · 1 item
- Open source blogs.nvidia.com ↗NVIDIA opens the 120B Nemotron 3.5 Super VL with datasets and training recipes; the 74B-A9B Bolt fits in 24GB with native 2-bit quantization
Take · In the 10-14 issue this was still a preview; today Bolt is actually on Hugging Face, and a 74B agent model runs in 24GB. I’ll test Bolt locally on PandaClaws’ batch rewriting to see how much 2-bit quantization costs on long Chinese text; Super VL, with its datasets and recipes, suits teams that want to fine-tune. Until the license is clear, evaluate only and keep it out of production.
From the Oct 16 brief · item 04 →
-
-
Oct 15 · Thu · 1 item
- Tools platform.claude.com ↗Claude API launches OpenTelemetry export in beta to trace Messages API requests and Managed Agents sessions
Take · I have been wrapping model calls in my own logging; that layer can now move to standard tracing. I’d start by adding an OpenTelemetry destination for document Q&A in the AI Cloud Drive, so one question shows as a single trace from file parsing to the model’s answer and I can see which step is slow; before production I’d check exactly what the spans carry, then decide which backend they go to.
From the Oct 15 brief · item 01 →
-
-
Oct 14 · Wed · 4 items
- Model release about.fb.com ↗Meta releases Muse Wonder and Muse Wonder Full for agentic tasks, with Muse Wonder available through the Meta Model API
Take · Meta is now a frontier API vendor, but the announcement gives no price and no benchmark numbers, and the safety report isn’t out yet. I’d register for the API and try one long agent task on it, but hold off on routing user data to it until the safety report and pricing are public; cross-conversation memory is exactly the feature that needs a clear data policy first.
From the Oct 14 brief · item 03 → - Open source huggingface.co ↗Liquid AI releases open-weight LFM2.5-24B-A2B, a reasoning agent model with 2B active parameters that decodes 184 tokens/s on an M5 Max
Take · Read the table, not the headline: it wins on tool calling and instruction following and loses clearly on coding and web tasks. That makes it a candidate for the local tool-routing and form-filling steps in PandaClaws, not for a coding agent. Given the 10-13 self-replication research, if you self-host it, keep the endpoint off the public internet.
From the Oct 14 brief · item 04 → - Tools github.blog ↗Copilot telemetry can attribute agent activity to a user without capturing prompts, with five content classes each off by default
Take · Separating identity from content is the right default: you can answer “whose agent ran this command” in an incident without storing everyone’s prompts. I’d enable identity plus tool arguments for audit, keep prompts and responses off, and apply the same split to the agent logs in our own products.
From the Oct 14 brief · item 07 → - Industry blogs.nvidia.com ↗NVIDIA lines up software for RTX Spark laptops shipping October 16, including the Nemotron 3.5 Bolt 74B agent model on October 15
Take · Together with LFM2.5 today, the local agent stack is filling in: hardware, small models, harnesses and llama.cpp support for decision models. I’d plan for “hybrid local and cloud routing” as a product feature, starting with keeping file parsing in the AI cloud drive on the user’s machine when it can, and only sending the hard questions out.
From the Oct 14 brief · item 09 →
-
-
Oct 12 · Mon · 1 item
- Open source huggingface.co ↗Amazon releases Strands decider models in 2B, 12B and 26B-A4B sizes under Apache 2.0
Take · Liquid AI, Cloudflare and now Amazon have shipped decision models within a week, so the category is real. For agent calls like which tool to use, whether arguments are valid and whether to block, I’d try the 2B version on routing in AI Interview first and, as the card advises, set confidence thresholds from my own traffic; the local server has no auth, so don’t expose it beyond the machine.
From the Oct 12 brief · item 03 →
-