A New Usability Inspection Method: Experience-Based Analysis

[Context and motivation] For software product development, user-based evaluation is often applied to detect usability issues. User-based testing is not cost-effective at every stage of the project, therefore for the early stage of the development, Usability Inspection Methods (UIM) are widely used where the evaluation is performed by experts. For the later phase, Summative Usability Testing (SUT) methods with end user involvement are used. However, UIMs have some drawbacks that make them complicated to use: finding good experts can be costly, the scope is too limited, and a lack of methods that allow for usability analysis before prototyping. This article introduces the Experience-based Analysis method (EbA) as an UIM that considers those drawbacks. [Question/problem] If we can obtain a considerable amount of usability information with EbA, we can use EbA as an effective UIM method. Therefore, the aim of the study is to identify to what extent the EbA can provide the same usability findings that is obtained by testing with users by SUT. [Results] An evaluation was done on a case study, where the information obtained by both EbA and SUT was analyzed by Content Analysis method. The results showed that most of the usability findings by SUT report were also identified by EbA. EbA method resulted in quantitatively more findings than SUT. [Contribution] The results lead us to recommend using EbA before SUT. As future work, we first plan to find whether the method gives any false positive or negative results. Second, we plan to conduct a survey to find the limitations of the method.