The Case Against Formal Verification, 50 Years Later

Engineers are getting excited astir package verification! This whitethorn travel arsenic a surprise, since verification has agelong been considered useful only successful very niche cases (at best; and impractical, useless aliases a complete discarded of clip astatine worst). Yet, the hype astir it is intelligibly here: Google Trends shows a ample spike successful searches for general verification/formal methods successful the past 2 years, everybody’s learning Lean, caller specification languages are popping up regularly, and location are efforts to verify awesome applications end-to-end (e.g., the Signal Shot project).

The main driver of this excitement is AI coding. First, AI agents time off a spread successful our knowing of the programs they write, frankincense creating the request for different intends of correctness assurance. Second, they make verification itself faster and easier to incorporated into real-world package development. Third, and possibly the astir important for business, if penning programs is made ace fast, each early gains will beryllium successful the area of package correctness assurance.

Will Wilson of Antithesis declares the triumph for this traditionally niche area successful his talk titled We won, what now? (The talk, delivered arsenic the opening to Bug Bash 2026, is great, and it gives immoderate bully ideas for the early of verification community, fixed the mainstream adoption.)

In that victorious context, it is absorbing to return to 1 of the classical papers arguing against general verification, Social Processes and Proofs of Theorems and Programs. Writing successful 1979, its authors say:

“We judge that (…) programme verification is bound to fail. We can’t spot really it’s going to beryllium capable to impact anyone’s assurance astir programs.”

I will spell done the arguments successful the insubstantial and analyse what caller developments (if any) invalidate them. This is simply a nosy workout alternatively than a wholly superior one: the insubstantial does not really declare that each general methods efforts are doomed (but only afloat verification). Furthermore, it is acold from clear that verification is going to go a regular portion of package engineering (what we are seeing are only early signs of interest). Nonetheless, re-examining successful 2026 obstacles that were seen arsenic basal 50 years agone will hopefully beryllium useful and interesting.

In this argument, the authors of the insubstantial reason against the thought that programming should go much for illustration mathematics successful position of each programme corresponding to a theorem that needs a proof. They say: Hold on, it is not the lawsuit moreover successful mathematics that proofs of theorems are the extremity of the process. Instead, the impervious is the first measurement and a intends of communication. The really important portion happens erstwhile different mathematicians internalize the proof, and the declare makes interaction pinch different branches of mathematics aliases beingness reality. This full process contributes to the claim’s believability.

Nothing to entity to here: proofs of programs do not request to correspond to mathematics exactly. (The statement is against a peculiar motivation, alternatively than against fundamentals of package verification.)

Argument 2: Problems pinch the specification

The first portion of the statement goes for illustration this: There is immoderate real-world request that is informal (the group progressive person a shared intuitive knowing of what the request is). This intuitive, informal request needs to beryllium translated into a general specification, which successful itself is an informal process. In that process, which is unverified, a batch tin beryllium mislaid aliases misinterpreted.

That is simply a adjacent point. The counterpoint is that specifications are person to informal requirements than implementations are (and frankincense a correction is easier to spot). Additionally, modern specification languages (such arsenic Quint) alteration examining the specification and each its edge-cases interactively, to make judge it really corresponds to our intuition.

The 2nd portion of the statement says that the specification is only valuable if it is independent of the implementation. Given the iterative quality of package development, that is almost impossible. Once the independency is lost, we are really conscionable aligning the specification and implementation (and perchance introducing to some akin mistakes).

I don’t deliberation this was a beardown statement moreover successful the past, and particularly not pinch coding agents successful the loop. Whenever further knowing is gained, this is bully wide for the improvement process. Humans, arsenic the last arbiters, determine which measurement to alteration the specification, re-examining first assumptions. Coding agents whitethorn beryllium allowed to nutrient and alteration code, and to nutrient proofs. If, however, location is simply a request to modify the specification, only a quality tin do that arsenic the last arbiter of what correct intends - which brings america backmost to the first portion of Argument 2.

Argument 3: Fully automatic verification is retired of reach

Having based on for why verification is bad arsenic a intends of communication, the authors displacement their attraction to the imaginable of afloat automatic verifiers (in which case, we could beryllium happy pinch the truth that a programme was proven correct, moreover if the impervious did not trigger a societal process among colleagues). Fully automatic verifiers, the authors argue, are very improbable to ever beryllium built.

In the meantime, location has been immoderate successful processing automatic verifiers, though quality effort (either penning proofs, aliases penning a suitable exemplary to beryllium exemplary checked) remains crucial. However, LLM-powered devices are closing this spread quickly. Igor Konnov, successful his station Formal proofs for distributed protocols pinch AI whitethorn beryllium person than you think, describes his acquisition proving the information of the Ben-Or protocol successful Lean.

Argument 4: Even if afloat automatic verification were wrong reach, it would beryllium detrimental

The authors declare that verifiers that simply respond pinch “VERIFIED” aliases “NOT VERIFIED” do not lend to knowing and would time off programmers clueless astir really to modify the programme further. Furthermore, they argue, having a verified programme whitethorn trim the inducement for different layers of defense (e.g., monitoring, rate-limiting and similar).

This is simply a anemic argument, relying connected the worst imaginable assumptions astir really verification devices and programmers’ behaviors would look successful the beingness of automatic verification.

Argument 5: Real-world systems are excessively messy to beryllium specified

There is simply a immense difference, it is correctly noted, betwixt algorithms and real-world systems. Whereas a specification for an algorithm tin often beryllium concise and tidy, specifications of real-world systems are ad-hoc, unstable, and messy. Furthermore, successful astir real-world systems, the algorithms are elemental and easy (and frankincense verifying them is not of awesome value).

It is existent that not each systems request to beryllium verified. However, successful past decades and years, location are changes that push for much verification:

• As package is entering captious infrastructure and the world of finance, the stakes get higher.
• If we person immoderate dream that a coding supplier creates what we want, we should amended beryllium capable to picture what we want. Of course, this does not ever request to beryllium a general specification, but the extremity of precisely specifying our intent becomes much important erstwhile we activity pinch coding agents. (This does not connote afloat verification, but the creation of specification excessively is 1 of the general methods tools.)

Argument 6: Software reliability is overmuch much than verification

“The desire to make programs correct is constructive and valuable. But the monolithic position of verification is unsighted to the benefits that could consequence from accepting a modular of correctness for illustration the modular of correctness for existent mathematical proofs, aliases a modular of reliability for illustration the modular for existent engineering structures. The quest for workability wrong economical limits, the willingness to transmission invention by recycling successful design, the spot successful the functioning of a organization of peers–all the mechanisms that make engineering and mathematics really activity are obscured successful the fruitless hunt for cleanable verifiability.”

I americium afloat onboard pinch this argument. Indeed, afloat verification of a strategy is seldom the champion measurement to spell astir reliability. All different efforts towards package correctness are arsenic valuable. And those 2 are not competitors: the accrued attraction connected champion methods to bring astir correctness is what matters.

## Conclusion This was rather a nosy insubstantial to read. The authors make a bully constituent that general verification is not a magic wand that solves each the correctness issues. Indeed, arsenic they constituent out, location is overmuch much to package correctness than verification: engineering processes, business considerations, further layers of defense, and truthful on.

Because of their attraction connected afloat verification, the authors of the insubstantial wrongly disregard really useful different parts of general methods are for wide understanding, amended creation choices, aliases higher velocity. All that gets amplified pinch AI coding agents penning the code, leaving humans pinch the task of specifying what needs to beryllium written and checking that it was written according to the fixed specification. This makes the activity of coding agents easier, too: verification gives them a measurement to adjacent the loop and show whether what they wrote was correct.

Thanks to 2 chap FM practitioners, Thomas Pani and Ranadeep Biswas, for useful discussions astir the insubstantial and this post. It would besides beryllium absorbing to perceive from group extracurricular the bubble who still find general methods useless.