Assessing the quality of diagnostic studies using psychometric instruments: applying QUADAS

Background There has been an increase in the number of systematic reviews of diagnostic tests, which has resulted in the introduction of two checklists: statement for reporting of diagnostic accuracy (STARD) and quality assessment of diagnostic accuracy studies (QUADAS). Objective To examine the validity and usefulness of QUADAS when applied to diagnostic accuracy studies using psychometric instruments and to examine the quality in reporting of these studies during practical application of the checklist. Method Two reviewers independently rated the quality of 54 studies using QUADAS. The proportion of agreement was used to assess overall agreement and individual agreement of QUADAS items between reviewers. Results The overall agreement between the two reviewers for all QUADAS items combined was 85.7%. The proportion of agreement between reviewers for each item ranged from just over 57-100% and was over 80% for 8 of the items. The poorest agreement was associated with the items for selection criteria, indeterminate results and withdrawals. None of the studies adequately reported all relevant information to enable all QUADAS item to be scored as 'yes'. Conclusion Overall QUADAS was relatively easy to use and appears to be an acceptable tool for appraising the quality of diagnostic accuracy studies using psychometric instruments. The application of QUADAS was hampered by the poor quality of reporting encountered.