Finding and Analyzing Compiler Warning Defects

Good compiler diagnostic warnings facilitate software development as they indicate likely programming mistakes or code smells. However, due to compiler bugs, the warnings may be erroneous, superfluous or missing, even for mature production compilers like GCC and Clang. In this paper, we (1) propose the first randomized differential testing technique to detect compiler warning defects and (2) describe our extensive evaluation in finding warning defects in widely-used C compilers. At the high level, our technique starts with generating random programs to trigger compilers to emit a variety of compiler warnings, aligns the warnings from different compilers, and identifies inconsistencies as potential bugs. We develop effective techniques to overcome three specific challenges: (1) How to generate random programs, (2) how to align textual warnings, and (3) how to reduce test programs for bug reporting? Our technique is very effective - we have found and reported 60 bugs for GCC (38 confirmed, assigned or fixed) and 39 for Clang (14 confirmed or fixed). This case study not only demonstrates our technique's effectiveness, but also highlights the need to continue improving compilers' warning support, an essential, but rather neglected aspect of compilers.