New AI system proves code safety with millions of lines of math

A major shift in software development is underway as artificial intelligence begins to handle the heavy lifting of mathematical proof.

Anthropic has partnered with Theorem to deploy its Claude model for a specific and demanding task: verifying that existing code is mathematically correct. This effort represents a significant step toward automating one of the most rigorous aspects of software engineering. The collaboration aims to make formal verification a routine part of the development cycle, similar to standard testing procedures. By leveraging AI, the team sought to reduce the time and expertise required to ensure software reliability.

The scale of the operation was substantial. In just a few weeks, the system generated approximately 180 million lines of proof code. This output covered 123 open-source libraries sourced from Google's OSS-Fuzz security project. The process did not involve writing new software from scratch. Instead, the model focused on analyzing existing code to confirm its correctness. It translated crash tests into mathematical problems and analyzed the compiled machine code to produce these extensive proofs.

The technical workflow involved several distinct stages. Claude would take the input from the open-source libraries and convert the testing scenarios into formal mathematical statements. It then analyzed the underlying machine code to verify if the code behaved as expected. An independent verification program named Rocq was used to check the validity of the generated proofs. According to Anthropic, this process required very little human intervention. The researchers describe the system as fully autonomous, although they note that the foundational tools and formal structures were built by human experts at Theorem.

The results included the discovery of hundreds of bugs that conventional testing had failed to catch. Many of these errors were hidden because they only manifested under very specific conditions. For example, some bugs required unusually large input files or a particular sequence of memory accesses to appear. Because these edge cases are rare in standard testing, they often slip through the net. The AI system was able to identify these issues and was even permitted to fix them automatically.

One notable example involved a widely used PDF reader. During the verification process, the team found as many security bugs in a single week as are typically reported for that software in an entire year. In another instance, Claude identified a logical error that had remained undetected for more than a decade. These findings suggest that even when a complete proof fails, the attempt itself can reveal significant flaws in the code. This capability offers a new layer of security for critical software components.

However, the method has clear limitations. Anthropic notes that a formally verified status does not mean the software is entirely error-free. The proofs are valid only under strict conditions. These include a specific processor architecture, single-threaded execution, and input sizes under 64 MB. In real-world applications, these libraries often run on different hardware and in more complex configurations. Therefore, the verification provides a strong guarantee, but it is not a universal one.

The computational cost of this process is high. Individual proofs can require hundreds of millions of tokens to complete. Anthropic has not disclosed the total cost of the project. However, the potential savings in the broader coding process could offset these expenses. If developers can trust AI-generated software more, they may spend less time on manual review. A smaller test run also indicated that proofs are reusable across different scenarios. Newer models are proving more efficient, with Opus 5.5 solving half the proofs using about 40 percent fewer tokens than its predecessor, Opus 5.

This project strengthens the argument for greater automation in software development. The central question is shifting from whether AI can write code to whether it can prove that code is safe. If the answer is yes, the impact could extend well beyond current coding tools. Anthropic expects that formal verification with AI will soon become as practical as standard testing. Theorem's roadmap already targets critical infrastructure components such as glibc, OpenSSL, and cURL, signaling a broader push for automated safety checks in essential software.