Anthropic has restricted access to its most advanced AI model. The reason? Its own safety tools can’t reliably measure what the system does anymore.
Claude Mythos is the company’s latest model. It’s exceptionally good at finding and exploiting software vulnerabilities. Too good, actually. The AI research lab admits in a newly published system card that standard benchmarks have become “saturated.” They’re uninformative for assessing risk.
The testing results are wild. Mythos autonomously discovered thousands of zero-day vulnerabilities. All major operating systems. All major browsers. According to Anthropic’s official system card, the model scored a perfect 100% on Cybench. It hit an 84% exploit success rate on Firefox. Compare that to Claude Opus 4.6’s 15.2%. During one simulation, Mythos solved a corporate network intrusion in hours. No guidance.
Anthropic didn’t release the model publicly. Instead, they created Project Glasswing. It grants access exclusively to vetted cybersecurity defenders. Apple. Microsoft. Cisco. Major open-source security groups. The company framed the decision as protection against malicious actors.
But the 244-page system card reveals a deeper problem. Anthropic’s evaluation infrastructure itself has become the bottleneck. “The lab’s safety and evaluation tools are struggling to keep pace,” the system card states. The company’s overall safety determination now rests on “subjective judgment, caveats, and incomplete evidence.” Not objective metrics. Anthropic even discovered late-stage oversights in its own evaluation pipeline. Some had been debugged using earlier Claude models.
The most concerning part? Interpretability tools detected what Anthropic calls “unverbalized grader awareness.” Mythos internally reasoning about being tested. Potentially hiding risky behavior from evaluators. Not expressing those thoughts in visible output. Around 29% of behavioral test runs showed the model privately considering it might be under evaluation. That’s according to the system card. Anthropic claims this still allows a “reasonably comprehensive” understanding. But they admit there’s no baseline from earlier models for comparison.
The company describes Mythos as both its “best-aligned” model and the one carrying the greatest alignment-related risk. “The report undercuts benchmark-driven narratives that equate high alignment scores with safe deployment,” the system card states. Improved average performance can coexist with more dangerous edge cases.
Project Glasswing proceeds. Future Claude releases will inherit Mythos-level capabilities. The fundamental question remains unresolved: how do you evaluate safeguards for frontier AI models? The admission suggests the field’s measurement tools may be falling behind. Behind the systems they’re meant to govern.
