Formal Verification: 180 Million Lines of Proof Generated by AI
Anthropic's collaboration with Theorem demonstrates that large language models can now mathematically validate existing software, uncovering latent bugs that traditional testing methods often miss.
By Thabo Mokoena ·
In a development that could reshape the landscape of software engineering, Anthropic’s Claude model generated approximately 180 million lines of mathematical proof code in a matter of weeks. This massive output was produced to verify the correctness of existing software libraries, marking a significant departure from traditional methods where such tasks required extensive human labor and time. The initiative, conducted in partnership with Theorem, represents a potential leap forward in how developers ensure the reliability of their code.
The core challenge addressed by this project is the historical expense associated with formal verification. Traditionally, proving that software runs without errors under all possible conditions has been prohibitively costly and labor-intensive. For context, a team of specialists required 20 person-years to formally verify the seL4 operating system kernel, a task previously reserved for high-stakes applications such as Airbus avionics. By leveraging AI, Anthropic aims to make this rigorous process more routine, similar to standard testing procedures.
Anthropic’s Frontier Red Team worked with Theorem to allow Claude to handle the proof generation autonomously. The model drew from test programs associated with 123 open-source libraries within Google’s OSS-Fuzz security project. Rather than writing new software, Claude translated crash tests into mathematical problems and analyzed compiled machine code. An independent verification program named Rocq then checked these proofs. According to Anthropic, the process requires minimal human intervention, a state the researchers describe as fully autonomous, though the underlying formal foundations were established by human experts at Theorem.
A significant byproduct of this verification process was the discovery of numerous software defects. During the proof generation, Claude identified hundreds of bugs that conventional testing had failed to catch. Many of these errors only manifested under highly specific conditions, such as unusually large input files or particular sequences of data access. In one notable instance, while verifying a widely used PDF reader, the team found as many security vulnerabilities in a single week as are typically reported for that software over an entire year. In another case, the model detected a logical error that had remained undetected for more than a decade.
Despite these successes, the project comes with strict practical limitations. Anthropic notes that the term formally verified does not equate to error-free in all contexts. The proofs are valid only under specific constraints, including a defined processor architecture, single-threaded execution, and input sizes under 64 megabytes. In real-world deployments, these libraries often operate on diverse hardware and configurations, meaning the verification does not cover every possible scenario. Additionally, the computational cost remains high, with individual proofs consuming hundreds of millions of tokens, though Anthropic has not disclosed the total project expenses.
The efficiency of the process appears to be improving as models evolve. A smaller test run indicated that proofs are reusable and that newer models are more token-efficient. Specifically, Opus 5.5 solved half of the proofs using approximately 40 percent fewer tokens than its predecessor, Opus 5. This trend suggests that as models become more efficient, the cost barrier to formal verification may decrease, making it more accessible for broader application in the software development lifecycle.
The implications of this work extend beyond immediate bug detection. If AI can both write code and guarantee its correctness under defined conditions, the industry’s focus may shift from questioning AI’s ability to program to validating its ability to prove safety. This could lead to greater trust in AI-generated software and reduced time spent on manual code review, saving costs across the entire development process.
Looking ahead, Theorem’s roadmap targets critical infrastructure components such as glibc, OpenSSL, and cURL for similar verification efforts. Anthropic anticipates that formal verification using AI will soon become as practical and common as standard testing. If these projections hold, the adoption of AI-driven verification could fundamentally alter how software is built, tested, and trusted in the coming years.
Source: The Decoder
