Interobserver variability in applying a radiographic definition for ARDS

Context: Acute lung injury (ALI) and ARDS are currently defined by the American-European Consensus Conference (IIE-CC) definition criteria, which contain a radiographic criterion. The accuracy or reliability of this consensus radiographic definition has not been evaluated, and no radiographic definition of ALI-ARDS has been evaluated by a large international group of experts. Objective: To study the interobserver variability in applying the AECC radiographic criterion for ALI-IIRDS. Design: Survey. Participants: A convenience sample of 21 experts selected from participants attending the 1997 Toronto Mechanical Ventilation Workshop and from members of the National Institutes of Health ARDS Network. Outcome measures: Participants reviewed 28 randomly selected chest radiograph from critically ill, hypoxemic (Pao(2)/fraction of inspired oxygen ratio, < 300) patients and decided whether the radiograph fulfilled the AECC definition for ALI-IIRDS. Results: Interobserver agreement in applying the AECC definition for ALI-ARDS was moderate (kappa = 0.55; 95% confidence interval, 0.52 to 0.57), Thirteen radiographs (43%) showed nearly complete agreement (defined as 20 or 21 readers in agreement). Nine radiographs (32%) had more than or equal to five dissenting readers. The percentage of radiographs interpreted as consistent with Al;I-ARDS by individual readers ranged from 36 to 71%. Participants commented that mild infiltrates, pleural effusions, atelectasis, isolated lower lobe involvement, radiographic technique, and overlying monitoring equipment posed the most difficulties. Conclusions: The radiographic criterion used in the current AECC definition for ALI-ARDS showed high interobserver variability when applied by expert investigators in the fields of mechanical ventilation and ARDS. This variability may result in differences in ALI-ARDS populations at different clinical research centers and may make it difficult for clinicians to apply the results of clinical trials to their patients. Modifications to the radiographic criterion or annotated reference radiograph may improve the reliability of future definitions for ALI-ARDS.