A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores

被引：117

作者：

De-Arteaga, Maria ^{[1
]}

Fogliato, Riccardo ^{[2
]}

Chouldechova, Alexandra ^{[3
]}

机构：

[1] Carnegie Mellon Univ, Heinz Coll, Machine Learning Dept, Pittsburgh, PA 15213 USA

[2] Carnegie Mellon Univ, Dept Stat & Data Sci, Pittsburgh, PA 15213 USA

[3] Carnegie Mellon Univ, Heinz Coll, Pittsburgh, PA 15213 USA

来源：

PROCEEDINGS OF THE 2020 CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS (CHI'20) | 2020年

关键词：

Human-in-the-loop; Decision support; Algorithm aversion; Automation bias; Algorithm assisted decision making; Child welfare; AUTOMATION BIAS; PERFORMANCE; FORECASTS; TRUST;

D O I：

10.1145/3313831.3376638

中图分类号：

TP3 [计算技术、计算机技术];

学科分类号：

0812 ;

摘要：

The increased use of algorithmic predictions in sensitive domains has been accompanied by both enthusiasm and concern. To understand the opportunities and risks of these technologies, it is key to study how experts alter their decisions when using such tools. In this paper, we study the adoption of an algorithmic tool used to assist child maltreatment hotline screening decisions. We focus on the question: Are humans capable of identifying cases in which the machine is wrong, and of overriding those recommendations? We first show that humans do alter their behavior when the tool is deployed. Then, we show that humans are less likely to adhere to the machine's recommendation when the score displayed is an incorrect estimate of risk, even when overriding the recommendation requires supervisory approval. These results highlight the risks of full automation and the importance of designing decision pipelines that provide humans with autonomy.

引用

页数：12

共 53 条

[1] The meta-analysis of clinical judgment project:: Fifty-six years of accumulated research on clinical versus statistical prediction [J].

AEgisdottir, S ;

White, MJ ;

Spengler, PM ;

Maugherman, AS ;

Anderson, LA ;

Cook, RS ;

Nichols, CN ;

Lampropoulos, GK ;

Walker, BS ;

Cohen, G ;

Rush, JD .

COUNSELING PSYCHOLOGIST, 2006, 34 (03) :341-382

[2]

Albright A., 2019, IF YOU GIVE JUDGE RI

[3]

Allegheny County Department of Health and Human Services, ALL FAM SCREEN TOOL

[4]

[Anonymous], 2012, Journal of Asynchronous Learning Networks, DOI DOI 10.24059/OLJ.V16I3.275

[5]

Bansal G., 2019, AAAI CONF ARTIF INTE, V33, P2429

[6]

Bastani Osbert, 2019, ARXIV PREPRINT ARXIV

[7] 'It's Reducing a Human Being to a Percentage'; Perceptions of Justice in Algorithmic Decisions [J].

Binns, Reuben ;

Van Kleek, Max ;

Veale, Michael ;

Lyngs, Ulrik ;

Zhao, Jun ;

Shadbolt, Nigel .

PROCEEDINGS OF THE 2018 CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS (CHI 2018), 2018,

[8]

Brown A, 2019, CHI 2019: PROCEEDINGS OF THE 2019 CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS, DOI [10.1145/3290605.3300271, 10.1080/15298868.2019.1664624]

[9] Sentencing Guidelines and Judicial Discretion: Quasi-Experimental Evidence from Human Calculation Errors [J].

Bushway, Shawn D. ;

Owens, Emily G. ;

Piehl, Anne Morrison .

JOURNAL OF EMPIRICAL LEGAL STUDIES, 2012, 9 (02) :291-319

[10] Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission [J].

Caruana, Rich ;

Lou, Yin ;

Gehrke, Johannes ;

Koch, Paul ;

Sturm, Marc ;

Elhadad, Noemie .

KDD'15: PROCEEDINGS OF THE 21ST ACM SIGKDD INTERNATIONAL CONFERENCE ON KNOWLEDGE DISCOVERY AND DATA MINING, 2015, :1721-1730

← 1 2 3 4 5 6 →