Bayesian Hypothesis Testing Illustrated: An Introduction for Software Engineering Researchers

被引:3
作者
Erdogmus, Hakan [1 ]
机构
[1] Carnegie Mellon Univ, Dept Elect & Comp Engn, 5000 Forbes Ave, Pittsburgh, PA 15213 USA
关键词
Bayesian statistics; Bayesian inference; Bayesian analysis; Bayesian hypothesis testing; frequentist analysis; frequentist inference; null hypothesis significance testing; NHST; empirical software engineering; software engineering research;
D O I
10.1145/3533383
中图分类号
TP301 [理论、方法];
学科分类号
081202 ;
摘要
Bayesian data analysis is gaining traction in many fields, including empirical studies in software engineering. Bayesian approaches provide many advantages over traditional, or frequentist, data analysis, but the mechanics often remain opaque to beginners due to the underlying computational complexity. Introductory articles, while successful in explaining the theory and principles, fail to provide a totally transparent operationalization. To address this gap, this tutorial provides a step-by-step illustration of Bayesian hypothesis testing in the context of software engineering research using a fully developed example and in comparison to the frequentist hypothesis testing approach. It shows how Bayesian analysis can help build evidence over time incrementally through a family of experiments. It also discusses chief advantages and disadvantages in an applied manner. A figshare package is provided for reproducing all calculations.
引用
收藏
页数:28
相关论文
共 50 条
[41]   Bayesian Networks For Evidence-Based Decision-Making in Software Engineering [J].
Misirli, Ayse Tosun ;
Bener, Ayse Basar .
IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, 2014, 40 (06) :533-554
[42]   Priors via imaginary training samples of sufficient statistics for objective Bayesian hypothesis testing [J].
Fouskakis, D. .
METRON-INTERNATIONAL JOURNAL OF STATISTICS, 2019, 77 (03) :179-199
[43]   Revisiting the Paleomagnetic Reversal Test: A Bayesian Hypothesis Testing Framework for a Common Mean Direction [J].
Heslop, D. ;
Roberts, A. P. .
JOURNAL OF GEOPHYSICAL RESEARCH-SOLID EARTH, 2018, 123 (09) :7225-7236
[44]   Bayesian hypothesis testing of four-taxon topologies using molecular sequence data [J].
Sinsheimer, JS ;
Lake, JA ;
Little, RJA .
BIOMETRICS, 1996, 52 (01) :193-210
[45]   A Bayesian actor-oriented multilevel relational event model with hypothesis testing procedures [J].
Vieira F. ;
Leenders R. ;
McFarland D. ;
Mulder J. .
Behaviormetrika, 2024, 51 (1) :37-74
[46]   A Bayesian hypothesis testing-based statistical decision philosophy for structural damage detection [J].
Zhang, Qiu-Hu ;
Ni, Yi-Qing .
STRUCTURAL HEALTH MONITORING-AN INTERNATIONAL JOURNAL, 2023, 22 (04) :2734-2754
[47]   Applying and Exploring Bayesian Hypothesis Testing for Large Scale Experimentation in Online Tutoring Systems [J].
Dommeti, Vijaya ;
Selent, Douglas .
PROCEEDINGS OF THE FOURTH (2017) ACM CONFERENCE ON LEARNING @ SCALE (L@S'17), 2017, :229-232
[48]   A UNIFIED CONDITIONAL FREQUENTIST AND BAYESIAN TEST FOR FIXED AND SEQUENTIAL SIMPLE HYPOTHESIS-TESTING [J].
BERGER, JO ;
BROWN, LD ;
WOLPERT, RL .
ANNALS OF STATISTICS, 1994, 22 (04) :1787-1807
[49]   Bayesian alternatives to null-hypothesis significance testing for repeated-measures designs [J].
Nathoo, Farouk S. ;
Masson, Michael E. J. .
JOURNAL OF MATHEMATICAL PSYCHOLOGY, 2016, 72 :144-157
[50]   Priors via imaginary training samples of sufficient statistics for objective Bayesian hypothesis testing [J].
D. Fouskakis .
METRON, 2019, 77 :179-199