Development and application of a mobile-based multimedia nursing competency evaluation system for nursing students: A mixed-method randomized controlled study

Aim: To develop a mobile-based multimedia Nursing Competency Evaluation (NCE) system based on the Attention, Relevance, Confidence, Satisfaction model and verify its effectiveness.Background: In education, mobile devices can enable the delivery of learning content without time and spatial constraints. Mobile-based test is emerging as a novel method using technologies to appraise students' perfor-mance on practicum. This mobile-based test go beyond the simple evaluation of memorised knowledge, a lim-itation common to paper-based tests. They are useful because they can include multimedia items such as videos, animations and pictures to comprehensively evaluate students' clinical competencies.Methods: This study was conducted in a nursing university in Seoul, South Korea in September 2021. A mixed method randomised controlled design was employed to evaluate its usability. The participants in the experi-mental group used the Nursing Competency Evaluation system and joined in focus group interviews for verifying the effects of the Nursing Competency Evaluation system qualitatively. Those in the control group responded to the mobile-based test, but which has only text-based test items. The system usability, effectiveness and learning satisfaction in both groups were measured after the mobile-based test experience. Quantitative and qualitative data were analysed using t-tests and thematic analysis using focus group interviews, respectively.Results: Sixty nursing students participated, with 30 each in the experimental and control groups. There were no significant differences in nursing competency scores between the two groups. However, average scores for effectiveness and learning satisfaction were significantly higher in the experimental than in the control group. Nineteen experimental group participants partook in the interviews, with many describing that the Nursing Competency Evaluation system allowed them to experience new learning contents and efficiently learn practical nursing skills that can be useful in clinical settings.Conclusions: The Nursing Competency Evaluation system is a promising method because it used mobile tech-nologies and multimedia to appraise students' performance on nursing practicum. It was found that the Nursing Competency Evaluation system with multimedia items is more realistic, interactive and satisfactory compared with text-based mobile test. Thus, we expect it to be used in future nursing curriculums to improve students' nursing competencies.