The Garbage Can Model: A Study in (Non)Reproducible Research

被引:0
作者
Levin, Stewart A. [1 ]
机构
[1] Stanford Univ, Stanford, CA 94305 USA
关键词
computational modeling; organizational behavior; reproducibility;
D O I
暂无
中图分类号
O1 [数学]; C [社会科学总论];
学科分类号
03 ; 0303 ; 0701 ; 070101 ;
摘要
The classic paper "A Garbage Can Model of Organizational Choice" by Cohen, March and Olsen (1972) both heralded a significant expansion of the study of organizational decision making to non-industrial settings, in particular universities, and served as a very early example of reproducible computational research, incorporating a Fortran 66 program in its appendix to permit others to reproduce their results and run further examples. In this work my extensive attempts to perfectly reproduce the original results show the inherent challenge of reproducing computational research in the presence of ever-changing computing platforms and technology. Indeed, exact values could not be reproduced in this study, nor any other published study, because of hypersensitivity. Despite this, additional statistics allowed by modern computer capabilities almost completely agree with the qualitative observations and conclusions in the original work. Finally, in light of the need for high precision, it will be worthwhile to retest and reevaluate later studies of the internal dynamics of the model that faulted the code for behavior at odds with the theory.
引用
收藏
页码:455 / 465
页数:11
相关论文
共 50 条
[31]   Neuroscout, a unified platform for generalizable and reproducible fMRI research [J].
de la Vega, Alejandro ;
Rocca, Roberta ;
Blair, Ross W. ;
Markiewicz, Christopher J. ;
Mentch, Jeff ;
Kent, James D. ;
Herholz, Peer ;
Ghosh, Satrajit S. ;
Poldrack, Russell A. ;
Yarkoni, Tal .
ELIFE, 2022, 11
[32]   An Invitation to Teaching Reproducible Research: Lessons from a Symposium [J].
Ball, Richard ;
Medeiros, Norm ;
Bussberg, Nicholas W. ;
Piekut, Aneta .
JOURNAL OF STATISTICS AND DATA SCIENCE EDUCATION, 2022, 30 (03) :209-218
[33]   Towards Reproducible Machine Learning Research in Information Retrieval [J].
Lucic, Ana ;
Bleeker, Maurits ;
de Rijke, Maarten ;
Sinha, Koustuv ;
Jullien, Sami ;
Stojnic, Robert .
PROCEEDINGS OF THE 45TH INTERNATIONAL ACM SIGIR CONFERENCE ON RESEARCH AND DEVELOPMENT IN INFORMATION RETRIEVAL (SIGIR '22), 2022, :3459-3461
[34]   Sharing practices of software artefacts and source code for reproducible research [J].
Jean-Quartier, Claire ;
Jeanquartier, Fleur ;
Stryeck, Sarah ;
Simon, Joerg ;
Soeser, Birgit ;
Hasani-Mavriqi, Ilire .
INTERNATIONAL JOURNAL OF DATA SCIENCE AND ANALYTICS, 2024,
[35]   GHTraffic: A Dataset for Reproducible Research in Service-Oriented Computing [J].
Bhagya, Thilini ;
Dietrich, Jens ;
Guesgen, Hans ;
Versteeg, Steve .
2018 IEEE INTERNATIONAL CONFERENCE ON WEB SERVICES (IEEE ICWS 2018), 2018, :123-130
[36]   A Principled Approach to Reproducible Research A Comparative Review Towards Scientific Integrity in Computational Research [J].
Sinha, Rajesh ;
Sudhish, Prem Sewak .
2016 IEEE INTERNATIONAL SYMPOSIUM ON ETHICS IN ENGINEERING, SCIENCE AND TECHNOLOGY (ETHICS), 2016,
[37]   Prediction scoring of data-driven discoveries for reproducible research [J].
Anna L. Smith ;
Tian Zheng ;
Andrew Gelman .
Statistics and Computing, 2023, 33
[38]   Reproducible research practices are underused in systematic reviews of biomedical interventions [J].
Page, Matthew J. ;
Altman, Douglas G. ;
Shamseer, Larissa ;
McKenzie, Joanne E. ;
Ahmadzai, Nadera ;
Wolfe, Dianna ;
Yazdi, Fatemeh ;
Catala-Lopez, Ferran ;
Tricco, Andrea C. ;
Moher, David .
JOURNAL OF CLINICAL EPIDEMIOLOGY, 2018, 94 :8-18
[39]   Reproducible Research for Scientific Computing: Tools and Strategies for Changing the Culture [J].
LeVeque, Randall J. ;
Mitchell, Ian M. ;
Stodden, Victoria .
COMPUTING IN SCIENCE & ENGINEERING, 2012, 14 (04) :13-17
[40]   On Reproducible Implementations in Unsupervised Concept Drift Detection Algorithms Research [J].
Lukats, Daniel ;
Stahl, Frederic .
ARTIFICIAL INTELLIGENCE XL, AI 2023, 2023, 14381 :204-209