Inter-observer reliability of radiological signs of necrotising enterocolitis in a population of high-risk newborns

Radiological examination of the abdomen is critical to the diagnosis of necrotising enterocolitis (NEC). Previous studies on the reproducibility of radiological findings have been limited in size and based only on infants with the disease. We conducted a study among a sample of high-risk infants with and without the diagnosis of NEC: (1) to measure the degree of inter-observer agreement of NEC radiological diagnosis and signs and (2) identify a profile of radiological signs which led the observers to make a diagnosis of NEC. We collected 297 X-rays from a sample of 57 newborns admitted in 1999 to neonatal intensive care and neonatal surgery units in Rome, Italy. Three specialists in paediatric radiology examined the films independently and without any clinical information about patients. The analyses were conducted on a total of 891 forms filled in by the observers. Kappa values were calculated to measure the inter-observer reliability. To identify the profiles of radiological signs, a multidimensional analysis, binary segmentation, was carried out. The reproducibility of radiographic signs was 0.55 (P < 0.01) for diffuse gaseous intestinal distention, 0.22 (P < 0.01) for bowel wall thickening, 0.10 (P < 0.01) for presence of portal venous gas and 0.29 (P < 0.01) for pneumatosis intestinalis. The agreement for radiographic diagnosis suspected/confirmed of NEC was 0.31 (P < 0.01). Among the 23 possible combinations of radiographic signs, the three radiologists indicated four profiles that produced a diagnosis of NEC containing, respectively, two, three, four and five signs. Our study found, in a large sample of radiographs selected from a population of infants with and without NEC, a poor reliability for NEC diagnosis and individual radiological signs among three expert radiologists. Clinical information and the presence of more than one radiological sign can reduce the margin of observer's error that inevitably exists when dealing with a diagnosis as difficult as NEC.