A Mixed-Method Usability Study on User Experience with Systematic Review Software

Systematic reviews are widely used in evidence-based medicine. Conducting a systematic review requires intensive mental efforts, especially during the study screening process. This challenge has motivated the development of intelligent software. This study examined and compared the performance, workload, and user experience of two systematic review tools – Colandr with Artificial Intelligence (AI) features and Covidence without AI features by conducting a mixed-method usability study. The results showed that reviewers had higher precision in citation screening using Colandr than using Covidence. However, the user experience with Colandr was not optimal due to problems in its user interface design. Therefore, we suggest that the design and development of AI-enabled SR software emphasize the usability of the interface and apply user-centered design principles. 84 Annual Meeting of the Association for Information Science & Technology | Oct. 29 – Nov. 3, 2021 | Salt Lake City, UT. Author(s) retain copyright, but ASIS&T receives an exclusive publication license.