Benchmarking Aided Decision Making in a Signal Detection Task

被引:43
作者
Bartlett, Megan L. [1 ]
McCarley, Jason S. [2 ]
机构
[1] Flinders Univ S Australia, Adelaide, SA, Australia
[2] Oregon State Univ, Sch Psychol Sci, 2950 Jefferson Way, Corvallis, OR 97331 USA
关键词
human-automation interaction; signal detection theory; decision-making strategies; contingent criterion model; LINEAR-MODELS; RESPONSE BIAS; AUTOMATION; PERFORMANCE; RELIABILITY; JUDGMENT; TRUST; UNCERTAINTY; INFORMATION; SENSITIVITY;
D O I
10.1177/0018720817700258
中图分类号
B84 [心理学]; C [社会科学总论]; Q98 [人类学];
学科分类号
03 ; 0303 ; 030303 ; 04 ; 0402 ;
摘要
Objective: A series of experiments examined human operators' strategies for interacting with highly (93%) reliable automated decision aids in a binary signal detection task. Background: Operators often interact with automated decision aids in a suboptimal way, achieving performance levels lower than predicted by a statistically ideal model of information integration. To better understand operators' inefficient use of decision aids, we compared participants' automation-aided performance levels with the predictions of seven statistical models of collaborative decision making. Method: Participants performed a binary signal detection task that asked them to classify random dot images as either blue or orange dominant. They made their judgments either unaided or with assistance from a 93% reliable automated decision aid that provided either graded (Experiments 1 and 3) or binary (Experiment 2) cues. We compared automation-aided performance with the predictions of seven statistical models of collaborative decision making, including a statistically optimal model and Robinson and Sorkin's contingent criterion model. Results and Conclusion: Automation-aided sensitivity hewed closest to the predictions of the two least efficient collaborative models, well short of statistically ideal levels. Performance was similar whether the aid provided graded or binary judgments. Model comparisons identified potential strategies by which participants integrated their judgments with the aid's. Application: Results lend insight into participants' automation-aided decision strategies and provide benchmarks for predicting automation-aided performance levels.
引用
收藏
页码:881 / 900
页数:20
相关论文
共 66 条
[1]   Effects of incorrect computer-aided detection (CAD) output on human decision-making in mammography [J].
Alberdi, E ;
Povyakalo, A ;
Strigini, L ;
Ayton, P .
ACADEMIC RADIOLOGY, 2004, 11 (08) :909-918
[2]  
Andre AD, 1998, HUM FAC ERG SOC P, P31
[3]  
[Anonymous], 2014, BAYESIAN COGNITIVE M, DOI DOI 10.1017/CBO9781139087759
[4]  
[Anonymous], 2013, Bayesian data analysis, third edition
[5]  
[Anonymous], 1988, Signal Detection Theory and Psychophysics
[6]   Optimally Interacting Minds [J].
Bahrami, Bahador ;
Olsen, Karsten ;
Latham, Peter E. ;
Roepstorff, Andreas ;
Rees, Geraint ;
Frith, Chris D. .
SCIENCE, 2010, 329 (5995) :1081-1085
[7]   A COMPONENTIAL ANALYSIS OF COGNITIVE EFFORT IN CHOICE [J].
BETTMAN, JR ;
JOHNSON, EJ ;
PAYNE, JW .
ORGANIZATIONAL BEHAVIOR AND HUMAN DECISION PROCESSES, 1990, 45 (01) :111-139
[8]  
Bisantz A. M., 2013, OXFORD HDB COGNITIVE, P579
[9]   HUMAN PROBABILITY MATCHING BEHAVIOR IN RESPONSE TO ALARMS OF VARYING RELIABILITY [J].
BLISS, JP ;
GILSON, RD ;
DEATON, JE .
ERGONOMICS, 1995, 38 (11) :2300-2312
[10]   User Settings of Cue Thresholds for Binary Categorization Decisions [J].
Botzer, Assaf ;
Meyer, Joachim ;
Bak, Peter ;
Parmet, Yisrael .
JOURNAL OF EXPERIMENTAL PSYCHOLOGY-APPLIED, 2010, 16 (01) :1-15