AI System Verifies Millions of Code Lines in Weeks, Finding Hidden Flaws

For developers and users relying on digital infrastructure, the assurance that software operates correctly without hidden errors has historically been a slow and costly process. A recent development suggests this landscape may shift, as an artificial intelligence model has successfully generated mathematical proofs for existing codebases, identifying defects that had persisted for years. This marks a significant step toward automating the rigorous verification of software safety.

The effort involved Anthropic’s Claude model working in partnership with Theorem, a specialist in formal verification. In a matter of weeks, the system produced approximately 180 million lines of proof code for 123 open-source libraries. These libraries were drawn from Google’s OSS-Fuzz security project. The scale of this output contrasts sharply with previous methods, which often required extensive manual labor. For instance, a team of specialists previously required 20 person-years to verify the seL4 operating system kernel, a task limited to critical applications like avionics due to its expense.

The process did not involve writing new software from scratch. Instead, Claude translated existing crash tests into mathematical problems and analyzed compiled machine code to generate extensive proofs. An independent tool named Rocq automatically checks the validity of these proofs. While Anthropic describes the generation as fully autonomous, the underlying formal foundations and tools were developed by human researchers at Theorem. The system also identified and corrected hundreds of bugs during the verification process, many of which only appeared under specific, uncommon conditions.

The practical impact of this detection capability is significant. In one instance, while verifying a widely used PDF reader, the team found as many security bugs in a single week as are typically reported for that software over an entire year. In another application, a logical error that had remained undetected for more than a decade was identified. Researchers noted that even when a complete proof fails, the attempt often surfaces real vulnerabilities, providing value beyond a simple pass or fail result.

However, the term formally verified carries specific technical limitations. The proofs hold only under strict conditions, including a specific processor architecture, single-threaded execution, and inputs smaller than 64 megabytes. Real-world deployments often vary in hardware and configuration. Additionally, the computational cost is high, with individual proofs consuming hundreds of millions of tokens. While total project costs were not disclosed, Anthropic notes that newer models are becoming more efficient; Opus 5.5 solved half the proofs with roughly 40 percent fewer tokens than its predecessor.

If AI systems can both write and verify code, the industry may move toward trusting automated software generation more broadly. This could reduce the time developers spend on manual review, lowering costs across the entire coding lifecycle. Theorem’s roadmap currently targets critical infrastructure components such as glibc, OpenSSL, and cURL. Anthropic anticipates that formal verification with AI will soon become as routine as standard testing, fundamentally changing how software safety is assured.