A Digital Twin-Based Multi-modal UI Adaptation Framework for Assistance Systems in Industry 4.0

As a consequence of digital transformation many aspects related to the industrial manufacturing processes are facing changes. In terms of Human-Machine Interaction, the User Interface (UI) plays the most important role as a mediator between the human and certain assistance systems. In traditional industrial environments, the UIs are usually designed to handle a unimodal input command (via touch screen, keyboard or mouse) and to present a feedback in a visual way. However, due to the nature of the tasks there is a need for the human workers to easily shift tasks and acquire new skills. For this reason, in the UI adaptation process the personal abilities and preferences of the human workers should be taken into consideration. In this paper, we present a novel reference model for multi-modal adaptive UIs for assistance systems in manufacturing processes. Our approach provides a solution framework for adaptation of assistance systems in manufacturing processes not only based on the environmental conditions, but also based on the personal characteristics and abilities of the human workers, obtained by a personalized Digital Twin.