Aperitif: Scaffolding Preregistrations to Automatically Generate Analysis Code and Methods Descriptions

被引:4
作者
Pang, Yuren [1 ]
Reinecke, Katharina [1 ]
Just, Rene [1 ]
机构
[1] Univ Washington, Paul G Allen Sch Comp Sci, Seattle, WA 98195 USA
来源
PROCEEDINGS OF THE 2022 CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS (CHI' 22) | 2022年
关键词
Preregistration; Experiment design; Reproducibility; Data analysis; PRE-REGISTRATION; TRANSPARENCY; REPLICATION; SUFFICIENT; NEITHER; GAME; NOR;
D O I
10.1145/3491102.3517707
中图分类号
学科分类号
摘要
The HCI community has been advocating preregistration as a practice to improve the credibility of scientific research. However, it remains unclear how HCI researchers preregister studies and what preregistration users perceive as benefits and challenges. By systematically reviewing the past four CHI proceedings and surveying 11 researchers, we found that only 1.11% of papers presented preregistered studies, though both authors and reviewers of preregistered studies perceive it as beneficial. Our formative studies revealed key challenges ranging from a lack of detail about the study design, hindering comprehensibility, to inconsistencies between preregistrations and published papers. To explore ways for addressing these issues, we developed Aperitif, a research prototype that scaffolds the preregistration process and automatically generates analysis code and a methods description. In an evaluation with 17 HCI researchers, we found that Aperitif reduces the effort of preregistering a study, facilitates researchers' workflows, and promotes consistency between research artifacts.
引用
收藏
页数:15
相关论文
共 94 条
  • [1] Adams Anne, 2008, QUALITITATIVE APPROA
  • [2] American Economic Association, 2012, AM EC ASS REG RAND C
  • [3] [Anonymous], 2021, US
  • [4] [Anonymous], 2021, HEROKU
  • [5] [Anonymous], 2019, 2019 CHI C HUM FACT, DOI DOI 10.1145/3290607.3310432
  • [6] [Anonymous], 2007, FDA AM ACT 2007 FDAA
  • [7] [Anonymous], ART REV BADG
  • [8] [Anonymous], 2020, REG REP MSR 2020
  • [9] Ball N., 2021, P 50 INT C ENV SYSTE, P1
  • [10] Berkeley Institute of Governmental Studies at the University of California, 2020, EV GOV POL