In this article we address the issue of imperfect judgment rankings in ranked-set sampling and, in particular, their effect on the properties of test procedures based on the ranked-set samples analog of the Mann-Whitney-Wilcoxon statistic, U(RSS). We consider the impact of these imperfect rankings on the null distribution of the statistic and the resulting effect on the nominal level of associated hypothesis tests. We propose a model for the probabilities of imperfect judgment rankings based on the concept of expected spacings and use this model to study the properties of tests based on U(RSS). This investigation includes both small-sample Monte Carlo power simulations and a detailed analysis of the asymptotic relative efficiency properties of the U(RSS) procedure. We also examine, as an indication of the merits of using ranked-set sampling, the relative cost of measuring the value of a sample item as compared to obtaining a judgment ordering of a set of sample items.