DEGENERACY AND DISCRETE RECEIVER OPERATING CHARACTERISTIC RATING DATA

Rationale and Objectives. Observer performance studies sometimes use too few cases for estimating diagnostic accuracy from binormal receiver operating characteristic (ROC) curves, One important problem is degenerate data sets. We compared a new algorithm, RSCORE4, with the exact-solution approach to degeneracy in ROCFIT and with the Wilcoxon statistic. Methods. Degenerate ROC solutions result from empty cells in the data matrix. We addressed this problem by adding a small constant to empty cells in a maximum-likelihood program, RSCORE4, When this method failed, the program branched to a pattern-search algorithm. We tested the program in a series of Monte Carlo studies. Results. RSCORE4 converged to nondegenerate solutions in every case and gave results closer to population values than ROCFIT or Wilcoxon. ROCFIT converged to exact-fit degenerate solutions, those with zero or infinite parameter values, in more than 40% of the samples. The Wilcoxon statistic was biased. Conclusion. RSCORE4 seems to outperform other currently recommended methods for dealing with degeneracy.