Four Utilities in Eyewitness Identification Practice: Dissociations Between Receiver Operating Characteristic (ROC) Analysis and Expected Utility Analysis

The present article focuses on a utility-based understanding of criminal justice practice regarding eyewitness identifications. We argue that there are 4 distinct types of utility that should be considered when evaluating an identification procedure. These include the utility associated with all identifications, the utility associated with only the high confidence identifications, the average utility across the full range of identifications, and the maximum utility that can be attained by selecting an ideal criterion. We show that in almost all cases in which the difference between 2 procedures is defined by a tradeoff between increased guilty suspect IDs and increased innocent suspect IDs, current ROC (receiver operating characteristic) curve approaches fail to provide unambiguous information about which eyewitness identification procedures are best in practice. We introduce a novel graphical technique called utility difference curves that illustrates the impact that differential assumptions about base rates and cost structures have on the likely benefits of different identification procedures. The research emphasizes the importance of considering assumptions about base rates and costs associated with different types of eyewitness errors. We also clarify situations in which the outcome of eyewitness experiments are unambiguous and those in which careful consideration of tradeoffs are necessary.