Anthropic's New AI Verification Push Finds Hidden Flaws in Major Open-Source Libraries

Anthropic and its partner Theorem have demonstrated a new approach that allows the Claude AI model to automatically verify the correctness of existing software. This development marks a significant step toward making formal verification a routine part of the software development lifecycle rather than a rare, specialized task. The system works by generating mathematical proofs that code behaves as expected, a process that has historically been too expensive and complex for widespread use.

The core of this innovation is the ability to handle massive volumes of verification work. In a recent effort, the team tasked Claude with analyzing 123 open-source libraries from Google's OSS-Fuzz security project. Over the course of just a few weeks, the model produced approximately 180 million lines of mathematical proof code. This volume of work highlights the scalability of using AI to manage the heavy lifting involved in formal verification, moving beyond the traditional reliance on small teams of human specialists.

Traditionally, proving that software runs without errors under all possible conditions has been prohibitively costly. For context, a previous effort to formally verify the small operating system kernel known as seL4 required 20 person-years of labor. Because of this high cost, such rigorous verification was typically reserved for critical systems like avionics in aircraft. The new AI-driven method aims to lower this barrier, making deep verification accessible for a much broader range of software applications.

Rather than writing new code from scratch, Claude focused on verifying existing programs. The model translated crash tests into mathematical problems and analyzed the compiled machine code to generate extensive proofs. These proofs are then checked by an independent verification program called Rocq. According to Anthropic, the process is largely autonomous, requiring very few human reviewers. While the researchers describe the proof generation as fully autonomous, the underlying formal foundations and tools were developed by humans at Theorem.

One of the most immediate benefits of this process is the discovery of hidden defects. While generating proofs, Claude identified hundreds of bugs that conventional testing had missed. Many of these issues only appear under specific conditions, such as unusually large input files or particular sequences of data access. In one notable case involving a widely used PDF reader, the team found as many security bugs in a single week as are typically reported for that software over an entire year.

The system also proved capable of catching long-standing logical errors. In one instance, Claude identified a flaw that had gone undetected for more than a decade. Even when a complete proof is not successfully generated, the process still surfaces real bugs. This suggests that the attempt at verification itself has value, providing a deeper level of scrutiny than standard testing protocols currently offer to developers.

However, Anthropic notes that formal verification does not guarantee that software is entirely error-free. The proofs hold under strict conditions, including specific processor architectures, single-threaded programs, and inputs under 64 MB. In real-world scenarios, these libraries often run on various hardware configurations and with different threading models. Therefore, the verification provides a strong baseline of correctness but does not cover every possible operating environment.

The computational cost of this process remains a significant factor. Individual proofs can require hundreds of millions of tokens to generate, though Anthropic has not disclosed the total project costs. Efficiency is improving, however. A smaller test run indicated that proofs are reusable, and newer models are becoming more efficient. For example, the Opus 5.5 model solved half the proofs using about 40 percent fewer tokens than its predecessor, Opus 5.

If AI can verify its own code, developers may place greater trust in AI-generated software, potentially reducing the time spent on manual review. This shift could lower costs across the entire coding process. The project strengthens the argument for software automation by addressing the question of safety. If the answer is that AI can prove code is safe under defined conditions, the implications could extend far beyond current coding tools.

Anthropic expects that AI-assisted formal verification will soon become as practical as standard testing. Theorem's roadmap already targets critical infrastructure components such as glibc, OpenSSL, and cURL. This expansion suggests a future where the assurance of software safety is integrated into the development workflow, driven by automated mathematical proofs rather than manual inspection alone.