Interactive Theorem Proving Modulo Fuzzing

Interactive theorem provers (ITPs) exploit the collaboration between humans and computers, enabling proof of complex theorems. Further, ITPs allow extraction of provably correct implementations from proofs. However, often, the extracted code interface with external libraries containing real-life complexities-proprietary library calls, remote/cloud APIs, complex models like ML models, inline assembly, highly non-linear arithmetic, vector instructions etc. We refer to such functions/operations as closed-box components. For such components, the user has to provide appropriate assumed lemmas to model the behavior of these functions. However, we found instances where these assumed lemmas are inconsistent with the actual semantics of these closed-box components. Hence, even correct-by-construction code extracted from an ITP may still behave incorrectly when interfaced with such closed-box components. To this end, we propose StarFuzz, that allows the F-star interactive theorem prover to provide better end-to-end assurance on the application-even when interfaced with the closed-box components. Under the hood, StarFuzz rides on Sadhak, an SMT solver that combines fuzz testing to allow satisfiability checking over closed-box components. On the F-star library that includes external implementations in OCaml, StarFuzz discovered four bugs-one bug that revealed an error on the assumed lemmas for a closed-box function, and three bugs in the external implementations of these components.