Randomized clinical trial of total vs. subtotal hysterectomy: validity of the trial questionnaire

Background. To ensure the internal validity of a trial it is recommended to undertake a validation study of the method measuring the outcome. Aims. To validate a self-administered, postal questionnaire (http://www.gyncph.suite.dk/praes/gimbel/gihtm) used for measuring the outcomes of a randomized clinical trial of total vs. subtotal abdominal hysterectomy. Methods. A study of the content validity (10 patients) and a combined qualitative and quantitative validation study (20 patients) were performed. Results. The median kappa of the 74 questions was 0.71 (range 0.22-1.00) and the median overall agreement was 0.76 (range 0.25-1.00). Five questions (7%) had kappa values below 0.40, 11 questions (15%) between 0.40 and 0.60, and 58 questions (78%) above 0.61. Three questions with a kappa value below 0.40 had an overall agreement of 0.75 or more. Significant logical inconsistencies were found in five questions, where the operation method was expected to be known. The qualitative validation resulted in a total of 80 comments. The number of comments for any given subject ranged from 0 to 12. The comments could be grouped and processed into four categories. Conclusions. The agreement was high. Inconsistencies regarding questions on the operation methods were found. Potential problems were identified in five questions (7%). The validation studies resulted in some changes to the questionnaire and we became aware of areas in which caution should be taken in the interpretation of the results of the randomized trial.