Oct 25, 2026 · Sun · 7 items

Copilot usage metrics split out by client, GitHub public monitoring adds validity checks and an API, OpenAI explains how it contains frontier training

Today's thread is visibility and containment: coding tools' model usage can be reconciled by client, leaked secrets show whether they still work, and a frontier lab is explaining in public how it fences in training and evaluation. For people building products, measurement and isolation come before wider rollout.

RSS
  1. The Copilot usage metrics API now breaks usage out by client, listing requests and tokens from Claude Code, Codex, OpenCode and other tools

    • The 1-day and 28-day reports for enterprises, organizations and their users gain a totals_by_client array. Client values include opencode, claude_code, codex, zed, copilot_cli, cloud_agent, code_review, vscode and jetbrains; unrecognized tools fall under other.
    • Each client gets successful model requests, sessions, prompt and output tokens and average tokens per request, broken down by model with a model_selection_mode of user-selected, auto or product-selected.
    • request_count counts model calls, not prompts, so one prompt can produce several. Clients are identified by the integration ID each request sends, which GitHub says is a best-effort grouping, not a verified identity.
    • The change is additive and existing fields are unchanged. It needs the Copilot usage metrics policy enabled, and is visible to enterprise owners, billing managers, organization owners and custom roles with View Copilot Metrics.

    Builder's takePut this next to the 10-24 change that named custom skills and MCP servers in the metrics: Copilot is turning into the shared model bill for several coding tools. I'd pull a 28-day report, see which client the tokens go to, and only then pick the team's default tool; just don't compare request_count with prompt counts, or treat client as audit evidence.

  2. GitHub public monitoring adds triage, continuous validity checks, a REST API and webhooks, so enterprises can go after leaked secrets that still work first

    • Enterprises can close alerts in the security dashboard as Dismissed or Fixed with a comment, or reopen them; an activity timeline keeps the audit trail, and alerts filter by repository, committer, secret type, attribution method and state.
    • Validity checks now run automatically and continuously for supported providers, and alerts show whether a credential is Active, Inactive or Unknown.
    • A new GET /enterprises/{enterprise}/secret-scanning/public-monitoring-alerts endpoint and an enterprise-level secret_scanning_public_monitoring_alert webhook send real-time notice when a new leak is attributed to the organization.
    • GitHub stresses that closing an alert only changes its state inside the enterprise: it does not revoke the credential or remove it from the public repository. It is in public preview for GitHub Enterprise Cloud, included in Advanced Security or Secret Protection licenses at no extra cost.

    Builder's takeThe LiteLLM poisoning on 10-24 showed how easily credentials leave laptops and CI; the still-valid column is exactly what tells you which key to rotate first. I'd wire this webhook into the existing on-call alerts, require same-day revocation for anything Active, and only let an alert be marked Fixed after the key is revoked.

  3. OpenAI describes how it contains frontier model training and evaluation, including offline environments and high-compute red-teaming

    • Published on October 23 under OpenAI's Security category, the post explains how OpenAI strengthened containment for frontier model training and evaluation.
    • The two measures named in OpenAI's summary are offline environments and high-compute red-teaming.

    Builder's takeRead together with the AISI review on 10-24, two frontier labs made containing the model during testing a public practice in the same week. We don't train frontier models, but the lesson carries: run agent evals and automated regression offline by default against fake external services, and allowlist any network access explicitly.

  4. Hugging Face launches a Trainium compatibility lookup: whether a Hub model runs unmodified on Trainium, with a script to run it yourself

    • The new torch-neuronx model support Space lists models by Selected, Trending or Downloads, and each result shows the core count, run time including compilation, and peak device memory.
    • Only Transformers text-generation and image-text-to-text models are tested for now, limited to models that fit on 4 Trainium2 devices (16 cores at most); diffusers, sentence transformers and more tasks are planned.
    • Each test generates 32 tokens: single-core runs are compared with CPU output, and sharded models are checked for fluent English by a small judge model. A PASS says nothing about throughput or longer generations.
    • Each model comes with its Neuron stack, packages and run.py, precompiled graphs can be synced from a public NEFF cache to skip compilation, and the underlying aws-neuron/torch-neuronx-model-support dataset can be queried with pandas.

    Builder's takeIf you want to move open models to Trainium to save money, the first step is now a lookup instead of trial and error. But it only proves a model starts: I'd take the PASS models through real requests, measure throughput and long outputs, and then compare against the current GPU bill.

  5. Tencent's WeChat Vision team open-sources WeVisOne-VL 2B and 4B under Apache 2.0: one model for detection, grounding, counting, OCR, segmentation and matting

    • Built on Qwen3-VL, it casts object-level tasks (open-vocabulary detection, referring localization, counting, pointing, OCR) and pixel-level tasks (referring segmentation, promptable alpha matting) as structured next-token prediction.
    • Pixel-level output comes from a lightweight decoder of about 9.6 million parameters; boxes and points use coordinate tokens from <0> to <1000>, and the model returns None when a target is absent.
    • For inference efficiency it emits structured output directly without chain-of-thought, and it currently handles single-turn images only, not video.
    • Tencent-authored code is released under Apache 2.0, with third-party components keeping their own licenses; it is positioned for research and not for autonomous high-impact decisions or identifying individuals.

    Builder's takeSearch-by-content and locating text in screenshots in an AI cloud drive still means chaining several models today, so this 4B model is worth a consolidation trial. I'd compare OCR localization and counting accuracy on my own image samples first; it doesn't handle video and explicitly shouldn't be used to identify people, so cross both off in the product design up front.

  6. GitHub merges notifications and Copilot in Microsoft Teams: mention @GitHub in a channel to go from a bug notification to a fix PR, with the old app retiring November 16

    • The new GitHub app delivers GitHub notifications in Teams channels, group chats and private chats; when a bug report arrives you mention @github, and it investigates, fixes the bug and opens a pull request for the team to review, test and merge.
    • The app was previously called GitHub notifications; teams that have it only need to update, and existing notification features keep working.
    • The legacy GitHub app is renamed GitHub (deprecated) and retires on November 16. To keep using Copilot in Teams, install the consolidated app and follow the prompts to connect an account and pick a repository the first time you mention @GitHub.

    Builder's takeHanding work to an agent straight from a notification is convenient, but it also means anyone who can mention it in the channel can make it touch a repository. I'd start in one low-risk repo's channel, be explicit about who may assign work, and keep PRs on normal review; Teams users should put the November 16 retirement on the calendar.

  7. Microsoft Research introduces Flint-interactive, letting agents produce interactive charts from an interaction spec instead of rounds of code and debugging

    • The authors name two problems with agent-generated interactive charts: the cost of repeated rounds of reasoning, coding and debugging, and reliability failures such as broken layouts and interactions.
    • Flint-interactive sits on top of Flint: its compiler translates user interactions into semantic events and high-level operators update the charts, so an agent only has to emit an interaction_spec alongside the Flint chart spec.
    • Users can have an agent render interactive charts in the conversation through Flint-MCP and MCP Apps, and developers can use the programming API so their own agent apps produce interactive charts.
    • The post reports no quantitative evaluation; the project site is microsoft.github.io/flint-chart, and interaction presets can be proposed on GitHub.

    Builder's takeHaving the model emit a spec instead of code is one of the cheapest ways to cut agent failure rates, and charts are just one case. I'd try Flint-MCP in a feature that produces reports for users and compare failure rate and token use against letting the model write front-end code; there's no published evaluation, so measure it yourself.

Researched and drafted with AI assistance; editorial standards and views set by Darius.