A mixed-methods evaluation framework for electronic health records usability studies

Background: Poor EHR design adds further challenges, especially in the areas of order entry and information visualization, with a net effect of increased rates of incidents, accidents, and mortality in ICU settings. Objective: The purpose of this study was to propose a novel, mixed-methods framework to understand EHR-related information overload by identifying and characterizing areas of suboptimal usability and clinician frustration within a vendor-based, provider-facing EHR interface. Methods: A mixed-methods, live observational usability study was conducted at a single, large, tertiary academic medical center in the Southeastern US utilizing a commercial, vendor based EHR. Physicians were asked to complete usability patient cases, provide responses to three surveys, and participant in a semi-structured interview. Results: Of the 25 enrolled ICU physician participants, there were 5(20%) attending physicians, 9 (36%) fellows, and 11 (44%) residents; 52% of participants were females. On average, residents were the quickest in completing the tasks while attending physician took the longest to complete the same task. Poor usability, complex interface screens, and difficulty to navigate the EHR significantly correlated with high frustration levels. Significant association were found between the occurrence of error messages and temporal demand such that more error messages resulted in longer completion time (p = .03). Discussion: Physicians remain frustrated with the EHR due to difficulty in finding patient information. EHR usability remains a critical challenge in healthcare, with implications for medical errors, patient safety, and clinician burnout. There is a need for scientific findings on current information needs and ways to improve EHRrelated information overload.