The Problems with Usability Testing

Most books discussing User Experience (UX) or usability often dedicate sections or entire chapters to an introduction to usability testing as a valid and reliable method to identify issues that occur within a given interface or system [e.g., 1, 3, 6, 10]. Indeed, there is an entire book written about specific methods and application of usability testing [e.g., 6]. Despite the plethora of books discussing usability testing, Rolf Molich and his colleagues have found variance in the usability issues found even amongst agency experts [e.g., 12, 13, 14]. These variances suggest that usability testing might often be inconsistently applied across the UX field. From our experience in industry, we put forth in this paper commonly problems with usability testing, which can result in the inconsistencies applied across the UX field. These problems are contrasted against suggested protocol found in literature, which provides a basis to support discussion of solutions and provide recommendations to new researchers.