Tools #Coding agents#AI agents Nov 18, 2026 · Originally published Nov 17

GitHub Copilot Evals enters public preview: use /eval to write repeatable tests for your own skills, custom agents and MCP tools

After changing a skill or MCP tool, you can now regression-test whether it got better or worse right inside Copilot. What it does, its engine and how to install it, for teams customizing their coding agents.

Image: GitHub Changelog
Primary source GitHub Changelog · github.blog Read the original ↗

// Key points

  • Ask Copilot in your own words or use the /eval command to create an eval for a skill, custom agent or MCP tool in your project; evals check both the result and how the agent got there.
  • You approve each run before it starts, see which checks passed or failed and why, and compare runs from before and after a change.
  • The evaluation engine is Vally, an extensible local eval tool.
  • Install it as a VS Code extension, a Copilot CLI plugin, or in the Copilot app via the github/evals marketplace; it is in public preview for Copilot Individual, Business and Enterprise.

Builder's takeThis is the one I wanted most. We have written a pile of skills for our coding agents and judge changes mostly by feel, which gets worse every time the model changes. I would write five eval cases for each of our three most-used skills and rerun them on every skill change or model swap, such as today's GPT-6.2 or MAI Code 2, keeping the results on file.

// Background · from #Coding agents

Full timeline →
  1. Nov 18 OpenAI releases GPT-6.2 Astra and GPT-6.2 Sol, with better professional outputs, coding, computer use and efficiency at lower cost
  2. Nov 18 Microsoft Ignite: MAI Code 2 and Code 2 Flash coding models, MAI Licensed Weights for trusted customers, and over 50 million agents in Agent 365
  3. Nov 18 Copilot code review adds Adaptive, which picks review effort per pull request by complexity and risk, and can flag exposed secrets during review

Every Monday, the week in AI in one email

Every Monday morning: the 6 AI stories that mattered most in the past seven days, with my take, in your inbox.

Your email is only used to send the weekly and is never shared. Every email has a one-click unsubscribe link. See the latest issue →