Assessment of user experience in video-based learning environments: From design guidelines to final product

To provide students with high-quality, tailored content, higher education institutions face the challenge of helping their instructors produce their own materials. New learning objects are usually assessed when students actually use the materials. To improve quality assurance in the design of learning objects, this paper proposes systemically analyzing the user experience of new learning objects. This involves not only considering student feedback, but also integrating other perspectives into a continuous feedback loop, through a participatory design approach. The proposed framework was tested during the implementation of a flipped classroom initiative in a large Chilean university over a period of six months, involving 6 content designers, 3 teachers, 2 teaching assistants and 98 students. We observe how elements of feedback are integrated through a systemic assessment, where the end result (the learning object) is contrasted with the elements that guided its creation (the design guidelines). In this way, the roles of designer and user are complemented so as to identify design gaps early on in the process of creating a solution. This study adds to the current literature by relating design guidelines with the principles of instruction during the design process in order to ensure the overall quality of the learning objects.