The role of hyperparameters in machine learning models and how to tune them

被引:16
作者
Arnold, Christian [1 ]
Biedebach, Luka [2 ]
Kuepfer, Andreas [3 ]
Neunhoeffer, Marcel [4 ,5 ]
机构
[1] Cardiff Univ, Dept Polit & Int Relat, Cardiff, Wales
[2] Reykjavik Univ, Dept Comp Sci, Reykjavik, Iceland
[3] Tech Univ Darmstadt, Inst Polit Sci, Darmstadt, Germany
[4] Boston Univ, Rafik B Hariri Inst Comp & Computat Sci & Engn, Boston, MA 02215 USA
[5] Ludwig Maximilians Univ Munchen, Dept Stat, Munich, Germany
关键词
Best Practice; Hyperparameter Optimization; Machine Learning; REPLICATION;
D O I
10.1017/psrm.2023.61
中图分类号
D0 [政治学、政治理论];
学科分类号
0302 ; 030201 ;
摘要
Hyperparameters critically influence how well machine learning models perform on unseen, out-of-sample data. Systematically comparing the performance of different hyperparameter settings will often go a long way in building confidence about a model's performance. However, analyzing 64 machine learning related manuscripts published in three leading political science journals (APSR, PA, and PSRM) between 2016 and 2021, we find that only 13 publications (20.31 percent) report the hyperparameters and also how they tuned them in either the paper or the appendix. We illustrate the dangers of cursory attention to model and tuning transparency in comparing machine learning models' capability to predict electoral violence from tweets. The tuning of hyperparameters and their documentation should become a standard component of robustness checks for machine learning models.
引用
收藏
页码:841 / 848
页数:8
相关论文
共 33 条
[1]  
Bergstra J, 2012, J MACH LEARN RES, V13, P281
[2]   Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges [J].
Bischl, Bernd ;
Binder, Martin ;
Lang, Michel ;
Pielok, Tobias ;
Richter, Jakob ;
Coors, Stefan ;
Thomas, Janek ;
Ullmann, Theresa ;
Becker, Marc ;
Boulesteix, Anne-Laure ;
Deng, Difan ;
Lindauer, Marius .
WILEY INTERDISCIPLINARY REVIEWS-DATA MINING AND KNOWLEDGE DISCOVERY, 2023, 13 (02)
[3]  
Bouthillier X., 2019, PMLR, P725
[4]  
Bouthillier X., 2021, P MACHINE LEARNING S, V3, P747
[5]   Using Word Order in Political Text Classification with Long Short-term Memory Models [J].
Chang, Charles ;
Masterson, Michael .
POLITICAL ANALYSIS, 2020, 28 (03) :395-411
[6]  
Chollet F., 2015, Keras
[7]  
Cooper A. F., 2021, Advances in Neural Information Processing Systems, V34, P3081
[8]   What Can We Learn from Predictive Modeling? [J].
Cranmer, Skyler J. ;
Desmarais, Bruce A. .
POLITICAL ANALYSIS, 2017, 25 (02) :145-166
[9]  
Fan X., 2020, PMLR, P2996
[10]   Enhancing Validity in Observational Settings When Replication is Not Possible [J].
Fariss, Christopher J. ;
Jones, Zachary M. .
POLITICAL SCIENCE RESEARCH AND METHODS, 2018, 6 (02) :365-380