Experiences in mixed reality-based collocated after action review

After action review (AAR) is a widely used training practice in which trainees and trainers review past training experiences and performance for the purpose of learning. AAR has often been conducted with video-based systems whereby a video of the action is reviewed afterward, usually at another location. This paper proposes collocated AAR of training experiences through mixed reality (MR). Collocated AAR allows users to review past training experiences in situ with the user’s current, real-world experience, i.e., the AAR is conducted at the same location where the action being reviewed occurred. MR enables a user-controlled egocentric viewpoint, augmentation such as a visual overlay of virtual information like conceptual visualizations, and playback of recorded training experiences collocated with the user’s current experience or that of an expert. Collocated AAR presents novel challenges for MR, such as collocating time, interactions, and visualizations of previous and current experiences. We created a collocated AAR system for anesthesia education, the augmented anesthesia machine visualization, and interactive debriefing system. The system enables collocated AAR in two applications related to anesthesia training: anesthesia machine operation training and skin disinfection training with a mannequin patient simulator. Collocated AAR was evaluated in two informal pilot studies by students (n = 19) and an educator (n = 1) not directly affiliated with the project. We review the anecdotal data collected from the studies and point toward ways to refine and improve collocated AAR.