A framework for evaluating reproducibility and replicability in economics

被引：4

作者：

Dreber, Anna ^{[1
,2
]}

Johannesson, Magnus ^{[1
]}

机构：

[1] Stockholm Sch Econ, Dept Econ, Stockholm, Sweden

[2] Univ Innsbruck, Dept Econ, Innsbruck, Austria

来源：

ECONOMIC INQUIRY | 2024年

关键词：

metascience; replicability; reproducibility; PUBLICATION BIAS; REPLICATION; CRISIS; TRANSPARENCY; CREDIBILITY; DICTATOR; POWER;

D O I：

10.1111/ecin.13244

中图分类号：

F [经济];

学科分类号：

02 ;

摘要：

We propose a framework for evaluating reproducibility and replicability in economics. Reproducibility is defined as testing if the results of an original study can be reproduced using the same data and replicability is defined as testing if the results of an original study hold in new data. We further divide reproducibility and replicability studies into five types: computational reproducibility, recreate reproducibility, robustness reproducibility, direct replicability and conceptual replicability. In addition to this typology we propose indicators to measure the degree of reproducibility and replicability in both individual studies and for a group of studies.

引用

页数：19

共 100 条

[91] Business and management research: Low instances of replication studies and a lack of author independence in replications [J].

Ryan, James C. ;

Tipu, Syed A. A. .

RESEARCH POLICY, 2022, 51 (01)

[92] Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results [J].

Silberzahn, R. ;

Uhlmann, E. L. ;

Martin, D. P. ;

Anselmi, P. ;

Aust, F. ;

Awtrey, E. ;

Bahnik, S. ;

Bai, F. ;

Bannard, C. ;

Bonnier, E. ;

Carlsson, R. ;

Cheung, F. ;

Christensen, G. ;

Clay, R. ;

Craig, M. A. ;

Dalla Rosa, A. ;

Dam, L. ;

Evans, M. H. ;

Cervantes, I. Flores ;

Fong, N. ;

Gamez-Djokic, M. ;

Glenz, A. ;

Gordon-McKeon, S. ;

Heaton, T. J. ;

Hederos, K. ;

Heene, M. ;

Mohr, A. J. Hofelich ;

Hogden, F. ;

Hui, K. ;

Johannesson, M. ;

Kalodimos, J. ;

Kaszubowski, E. ;

Kennedy, D. M. ;

Lei, R. ;

Lindsay, T. A. ;

Liverani, S. ;

Madan, C. R. ;

Molden, D. ;

Molleman, E. ;

Morey, R. D. ;

Mulder, L. B. ;

Nijstad, B. R. ;

Pope, N. G. ;

Pope, B. ;

Prenoveau, J. M. ;

Rink, F. ;

Robusto, E. ;

Roderique, H. ;

Sandberg, A. ;

Schlueter, E. .

ADVANCES IN METHODS AND PRACTICES IN PSYCHOLOGICAL SCIENCE, 2018, 1 (03) :337-356

[93] False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant [J].

Simmons, Joseph P. ;

Nelson, Leif D. ;

Simonsohn, Uri .

PSYCHOLOGICAL SCIENCE, 2011, 22 (11) :1359-1366

[94] Specification curve analysis [J].

Simonsohn, Uri ;

Simmons, Joseph P. ;

Nelson, Leif D. .

NATURE HUMAN BEHAVIOUR, 2020, 4 (11) :1208-1214

[95] Small Telescopes: Detectability and the Evaluation of Replication Results [J].

Simonsohn, Uri .

PSYCHOLOGICAL SCIENCE, 2015, 26 (05) :559-569

[96] Increasing Transparency Through a Multiverse Analysis [J].

Steegen, Sara ;

Tuerlinckx, Francis ;

Gelman, Andrew ;

Vanpaemel, Wolf .

PERSPECTIVES ON PSYCHOLOGICAL SCIENCE, 2016, 11 (05) :702-712

[97] Publication bias: evidence of delayed publication in a cohort study of clinical research projects [J].

Stern, JM ;

Simes, RJ .

BMJ-BRITISH MEDICAL JOURNAL, 1997, 315 (7109) :640-645

[98] Leveraging vibration of effects analysis for robust discovery in observational biomedical data science [J].

Tierney, Braden T. ;

Anderson, Elizabeth ;

Tan, Yingxuan ;

Claypool, Kajal ;

Tangirala, Sivateja ;

Kostic, Aleksandar D. ;

Manrai, Arjun K. ;

Patel, Chirag J. .

PLOS BIOLOGY, 2021, 19 (09)

[99] Bayesian inference for psychology. Part II: Example applications with JASP [J].

Wagenmakers, Eric-Jan ;

Love, Jonathon ;

Marsman, Maarten ;

Jamil, Tahira ;

Ly, Alexander ;

Verhagen, Josine ;

Selker, Ravi ;

Gronau, Quentin F. ;

Dropmann, Damian ;

Boutin, Bruno ;

Meerhoff, Frans ;

Knight, Patrick ;

Raj, Akash ;

van Kesteren, Erik-Jan ;

van Doorn, Johnny ;

Smira, Martin ;

Epskamp, Sacha ;

Etz, Alexander ;

Matzke, Dora ;

de Jong, Tim ;

van den Bergh, Don ;

Sarafoglou, Alexandra ;

Steingroever, Helen ;

Derks, Koen ;

Rouder, Jeffrey N. ;

Morey, Richard D. .

PSYCHONOMIC BULLETIN & REVIEW, 2018, 25 (01) :58-76

[100]

Zahra T., 2023, Replication study of Baron (2022): school spending and student outcomes

← 1 2 3 4 5 6 7 8 9 10 →