A cognitive assistance system with augmented reality for manual repair tasks with high variability based on the digital twin

Cognitive assistance systems with augmented reality are a promising solution to the increasing complex-ity in all industries. Our proposed assistance system is geared towards manual repair tasks with high vari-ability in the maintenance, repair and overhaul industry. We use the digital twin for object-specific information and a human-machine interface that uses a web application for digital work instructions and augmented reality for spatial information. Context awareness of the assistance system is achieved by using two stationary 3D cameras and a barcode scanner. Camera images and point cloud data are used for precise markerless pose estimation and body tracking. (c) 2022 The Author(s). Published by Elsevier Ltd on behalf of Society of Manufacturing Engineers (SME). This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).