A Comparative Study of Heuristic Evaluation and Cognitive Walkthrough: An E-Government Usability Assessment Case Study

Ensuring usability is essential in software systems, and various testing methods are employed for this purpose. Expert-based inspection methods, like Cognitive Walkthrough (CW) and Heuristic Evaluation (HE), are widely used to identify usability issues. However, these methods can be time-consuming, highlighting the need to assess their effectiveness in usability evaluation. The objective of this study is to conduct a comparative analysis of two expert-based usability evaluation methods, CW and HE, as applied to Absher's website and mobile application, the most widely used governmental system in Saudi Arabia. Five independent evaluators applied these methods, and an in-depth comparison was made on the identified usability issues, their severity, and the corresponding usability attributes. HE identified a total of 83 issues, while CW identified 58, with HE being more effective at detecting major usability problems and CW excelling in identifying catastrophic issues. In terms of usability attributes, HE identified more issues related to effectiveness, efficiency, and errors, while CW detected a higher number of learnability problems. Both methods performed similarly in identifying memorability issues. This study offers insights into the strengths and limitations of both CW and HE, contributing valuable guidance on selecting the appropriate usability evaluation method based on the system's context.