A distribution-free procedure for comparing receiver operating characteristic curves from a paired experiment

A distribution-free permutation test procedure is proposed for comparing receiver operating characteristic curves based on continuous data from a paired design. The method tests the hypothesis that the two curves are identical for all operating points, unlike previously Proposed methods which test the equivalence of the areas under the curves. The new test is shown by simulation to have very similar operating characteristics to the standard method based on comparisons of the areas when the curves are parallel, but markedly superior power when the curves cross, that is when the curves are different but have similar areas. The prospects of generalising the approach to unpaired experiments and to comparisons of ordinal rating data are discussed.