The "perfect" reader study

A reader study, i.e. a diagnostic accuracy study aiming to assess clinical performance of one technology versus another, on the basis of image interpretation by a group of human readers, requires an accurate design, with the precise definition of each element concurring to the study itself. In this paper, the main issues to take into consideration when designing reader studies are described, and the characteristics of a number of indices of diagnostic accuracy are summarized. Special care is given to the description of the multiple sources of bias, which should be controlled in order to assure study soundness and allow consistent conclusions. Although a "perfect" reader study does not exist, because bias cannot be completely eliminated, a good study would be designed to minimize the bias effects, and possible unavoidable bias should be described in study reporting and used to provide a correct interpretation of results.