The researcher and the consultant: a dialogue on null hypothesis significance testing

Since its introduction, null hypothesis significance testing (NHST) has caused much debate. Many publications on common misunderstandings have appeared. Despite the many cautions, NHST remains one of the most prevalent, misused and abused statistical procedures in the biomedical literature. This article is directed at practicing researchers with limited statistical background who are driven by subject matter questions and have empirical data to be analyzed. We use a dialogue as in ancient Greek literature for didactic purposes. We illustrate several, though only a few, irritations that can come up when a researcher with minimal statistical background but a good sense of what she wants her study to do, and of what she wants to do with her study, asks for consultation by a statistician. We provide insights into the meaning of several concepts including null and alternative hypothesis, one- and two-sided null hypotheses, statistical models, test statistic, rejection and acceptance regions, type I and II error, p value, and the frequentist’ concept of endless study repetitions.