GitHub and Microsoft scan six open source projects with AI: thousands of possible issues become 99 private reports, and 55 were fixed by November 15
AI makes finding vulnerabilities fast and cheap; the hard part is getting them fixed. How one pilot moved from dumping findings on maintainers to sending only verified reports with suggested fixes, for people building security scanners or maintaining open source.
// Key points
- Microsoft's multi-model agentic scanning harness, Codename MDASH, used the MAI Cyber model to scan six projects (AutoGPT, Caddy, Express, FastMCP, ImageMagick and Netty), all alumni of the GitHub Secure Open Source Fund, with their maintainers' agreement.
- The pipeline: MDASH flagged thousands of possible issues with suggested fixes; an AI triage agent from GitHub Security Lab removed duplicates and checked validity; seven Microsoft FORGE Lab researchers verified every remaining finding by hand; reports went out through private vulnerability reporting with a proof of concept and a fix.
- Maintainers received 99 reports; as of November 15 they had fixed 55, and 14 of those fixes were already in a release.
- The first fixes shipped about a week after the initial scan; Caddy fixed all 22 of its reports in a week or less.
Builder's takeThousands down to 99 is the lesson in itself: the value of AI scanning is in filtering and verifying, not volume. When we wire up AI security scanning, I would require every result to come with reproduction steps and a patch, or not come at all. FastMCP is on the list, so teams building MCP servers on it should check recent release notes and make sure they have upgraded.