Actively Viewing the Audio-Visual Media: An Eye-Controlled Application for Experience and Learning in Traditional Chinese Painting Exhibitions

Traditional unidirectional communication through audio-visual media, such as video and slide, is a universal method of vividly narrating artifacts in museums. This traditional method is evolving with the integration of new technologies to engage visitors, enhancing their experience and learning. In recent years, eye-controlled interaction has been used to enrich traditional exhibitions, especially of ancient paintings. This article presents an eye-controlled application for actively viewing the audio-visual media of a traditional Chinese painting. Users can trigger animations and audio narrations by focusing their gaze on specific areas of the screen. To evaluate the value of experience and learning in integrating audio-visual media with eye-controlled interaction, we compared Eye-Controlled Application with the two baseline conditions (Video and Slide) through a comparison experiment (N = 40) in terms of user experience, learning outcome, cognitive load, intrinsic motivation, and visual behavior. The findings indicated the positive effects of Eye-Controlled Application on experience and learning, especially understanding. Users had stronger intrinsic motivation using Eye-Controlled Application, which contributed to positive learning outcomes. We found that Eye-Controlled Application changed visual behavior by increasing the proportion of gaze patterns and decreasing the proportion of movement patterns and scan patterns. The alteration in visual behavior primarily stemmed from the eye-controlled interaction, which directed the user's focus toward specific areas, thereby guiding their attention to the necessary content. Additionally, the qualitative results suggested that each of the three conditions had its advantages and disadvantages. Importantly, the condition with Eye-Controlled Application carried potential risks of distraction, reduced efficiency, and increased burden. This study contributes to the potential of eye-controlled interaction for enhancing visitors' experience and learning, providing new insights into the development of interactive audio-visual media within museums.