Impact of observer variability on the usefulness of endoscopic images for the documentation of upper gastrointestinal endoscopy

Objective. Endoscopy is an observer-dependent diagnostic method, which, until recently, has lacked precise guidelines for written reports. There is an increasing demand for improvement in endoscopy records, which may necessitate the supplementation of image documentation. The aim of this study was to estimate interobserver as well as intra-observer variability in the assessment of images from gastroscopy. Material and methods. We designed an Internet interface presenting endoscopy images, accompanied by a multiple-choice questionnaire for assessing pathology in the images. Ten images from the distal oesophagus and 10 images from the pyloric antrum were chosen. In order to study interobserver variability, physicians with varying endoscopy experience were invited to complete the questionnaire. The physicians were re-invited 5 months later to assess the same images again, this time in order to assess intra-observer variability. Kappa statistics were used for analysis of agreement. Results. Initially, 13 of 20 invited physicians responded. Interobserver agreement varied between poor ( k<0.2) and moderate ( 0.4<k<0.6). In the second part of the study, 10 of 11 invited physicians responded. Intra-observer agreement varied between moderate ( 0.4<k<0.6) and good ( 0.6<k<0.8). A higher level of experience does not imply either better interobserver or better intra-observer agreement. Images of concise endoscopy findings, such as the presence of an ulcer, resulted in better agreement than did the assessment of images of less definable findings. Conclusion. The variability in the interpretation of endoscopy images is large. We therefore believe that systematic inclusion of a set of images into endoscopy reports will improve their quality.