Assessment of adverse events in medical care: lack of consistency between experienced teams using the global trigger tool

Background: Many patients are harmed as the result of healthcare. A retrospective structured record review is one way to identify adverse events (AEs). One such review approach is the global trigger tool (GTT), a consistent and well-developed method used to detect AEs. The GTT was originally intended to be used for measuring data over time within a single organisation. However, as the method spreads, it is likely that comparisons of GTT safety outcomes between hospitals will occur. Objective: To evaluate agreement in judgement of AEs between well-trained GTT teams from different hospitals. Methods: Five teams from five hospitals of different sizes in the southeast of Sweden conducted a retrospective review of patient records from a random sample of 50 admissions between October 2009 and May 2010. Inter-rater reliability between teams was assessed using descriptive and kappa statistics. Results: The five teams identified 42 different AEs altogether. The number of identified AEs differed between the teams, corresponding to a level of AEs ranging from 27.2 to 99.7 per 1000 hospital days. Pair-wise agreement for detection of AEs ranged from 88% to 96%, with weighted kappa values between 0.26 and 0.77. Of the AEs, 29 (69%) were identified by only one team and not by the other four groups. Most AEs resulted in minor and transient harm, the most common being healthcare-associated infections. The level of agreement regarding the potential for prevention showed a large variation between the teams. Conclusions: The results do not encourage the use of the GTT for making comparisons between hospitals. The use of the GTT to this end would require substantial training to achieve better agreement across reviewer teams.