Development and validation of a new patient experience tool in patients with serious illness

Background: Patients with serious chronic illnesses face increasingly complex care and are at risk of poor experience due to a fragmented health system. Most current patient experience tools are not designed to address the unique care aspects of this population and the few that exist are delivered too late in the disease trajectory and are not administered longitudinally which makes them less useful across settings. Methods: We developed a new tool designed to address these gaps. The 25 item scale was tested and refined using randomly cross-validated exploratory and confirmatory factor analyses. Participants were not yet hospice eligible but sick enough to receive benefits of a supportive care approach in the last 2 to 3 years of life. Full information maximum likelihood models were run to confirm the factor structure developed in exploratory analyses. Goodness-of-fit was assessed with the Comparative Fit Index, the Tucker-Lewis Index, and the Root Mean Square Error of Approximation. Test-retest reliability was assessed with the intraclass correlation coefficient and internal consistency of the final scale was examined using Cronbach's alpha. Results: Exploratory factor analysis revealed three domains - Care Team, Communication, and Care Goals - after removing weak loading and cross loading items. The initial three domain measurement model suggested in the development cohort was tested in the validation cohort and exhibited poor fit X-2 (206) = 565.37, p < 0.001; CFI = 0.879; TLI = 0.864; RMSEA = 0.076. After model respecification, including removing one additional item and allowing paths between theoretically plausible error terms, the final 21 item tool exhibited good fit X2 (173) = 295.63, p < 0.001; CFI = 0.958; TLI = 0.949; RMSEA = 0.048. Cronbach's alpha revealed high reliability of each domain (Care Team = 0.92, Communication = 0.83, Care Goals = 0.77) and the entire scale (alpha = 0.91). ICC showed adequate test-retest validity (ICC = 0.58; 95% CI: 0.52-0.65) of the full scale. Conclusions: When administered earlier in the chronic illness trajectory, a new patient experience scale focused on care teams across settings, communication, and care goals, displayed strong reliability and performed well psychometrically.