The impact of data-analysis methods on cumulative research knowledge - Statistical significance testing, confidence intervals, and meta-analysis

The methods of data analysis used in research have a major effect on the development of cumulative knowledge. Traditional methods based on statistical significance testing have systematically retarded the growth of cumulative research knowledge by making ii virtually impossible to discern the real meaning of research literatures. Meta-analysis makes it possible to demonstrate graphically the high price the research enterprise has paid for its reliance on significance testing. Beet in addition to these demonstrations, reform will require that researchers come to understand that the benefits they see as flowing from the use of significance tests are illusory In research practice and in training of researchers we must use and teach appropriate data analysis methods: point estimates of effect sizes and confidence intervals within individual studies, and meta-analysis in the integration of multiple studies to produce final conclusions. These reforms are essential to the progress of cumulative research knowledge.