QuickChecking static analysis properties

Summary A static analysis can check programs for potential errors. A natural question that arises is therefore: who checks the checker? Researchers have given this question varying attention, ranging from basic testing techniques, informal monotonicity arguments, thorough pen‐and‐paper soundness proofs, to verified fixed point checking. In this paper, we demonstrate how quickchecking can be useful to test a range of static analysis properties with limited effort. We show how to check a range of algebraic lattice properties, to help ensure that an implementation follows the formal specification of a lattice. Moreover, we offer a number of generic, type‐safe combinators to check transfer functions and operators on lattices, to help ensure that these are, eg, monotone, strict, or invariant. We substantiate our claims by quickchecking a type analysis for the Lua programming language. The combination of QuickCheck (property‐based testing) with (1) a reusable test suite of common algebraic properties and (2) a domain‐specific language of test signatures is effective for locating errors in a static analysis implementation and for increased test coverage of the underlying source code.