Usability evaluation of order sets in a computerised provider order entry system

Background: Computerised provider order entry (CPOE) is an important patient safety intervention that has encountered significant barriers to implementation. The usability of a CPOE system plays a significant role in its acceptance. The authors conducted a heuristic evaluation of a CPOE order set system to uncover existing usability issues prior to implementation. Methods: A heuristic evaluation methodology was used to evaluate the usability of a CPOE test order set system. There are 10 heuristic principles, such as error prevention, to help users identify and recover from errors. Evaluators included a staff physician with extensive clinical experience, and three engineers with expertise in heuristic evaluation methodology. The results of the heuristic evaluation were used to create a user centred design prototype. Results: 92 unique heuristic violations were found for the CPOE test order set system, including 35 identified by the clinician and at least one engineer, and 57 of the 92 violations (62%) found only by the clinician. All evaluators identified at least one violation of each of the 10 usability heuristics in their analysis of the CPOE system. A user centred design prototype was created to demonstrate changes that could improve usability. Interpretation: The CPOE test order set system had many usability heuristic violations. Many violations were found by a clinician with knowledge of the heuristic evaluation process. Implementation of the CPOE system was deferred and a new user centred design prototype was developed for future study. The authors recommend conducting heuristic evaluations early in the process of designing, selecting and implementing CPOE systems.