AI-based Interactive Digital Assistants for Virtual Reality in Educational Contexts

This study presents and investigates a novel use of Artificial Intelligence (AI) for the creation of digital assistants in Virtual Reality (VR) environments for training and educational contexts. The concept proposed in this work couples two separate AI systems, the Movement Model (MM) and the Language Model (LM), to generate both conversational and visual responses simultaneously. The LM uses a two channel system to simplify communication with the MM, such that it responds to the user through one channel, and provides input to the MM through the other. This input allows the MM to produce context-relevant movements to assist communication with the user. Additionally, since this process is handled automatically by AI systems, it can provide a uniquely simple method for the creation and customization of digital assistants, with minimal technical knowledge or time investment from the user. A proof-of-concept prototype was implemented and underwent preliminary validation via a comparison to the Avatar Replay System (ARS) from our previous work in a user study. It was found that the AI assistants were able to interact with the users comparably to human recordings captured with the ARS assistants.