Machine learning for assistance systems: pattern-based approach to online step recognition

Computer-aided assistance systems are entering the world of work and production. Such systems utilize augmented- and virtual-reality for operator training and live guidance as well as mobile maintenance and support. This is particularly important in the modern production reality of ever-changing products and 'lot size one' customization of production. This paper focuses on the application of machine learning approach to extend the functionality of assistance systems. Machine learning provides tools to analyse large amounts of data and extract meaningful information. The goal here is to recognize the movement of an operator which would enable automatic display of instructions relevant to them. We present the challenges facing machine learning applications in human-centered assistance systems and a framework to assess machine learning approaches feasible for this scenario. The approach is assessed on a historical data set and then deployed in a work station for live testing. The post-hoc, or historical, analysis yields promising results. The ad-hoc, or live, analysis is a complex task and the results are affected by multiple factors, most of which are introduced by the human influence. The contribution of this paper is an approach to adapt state-of-the-art machine learning to operator movement recognition with a special focus on approaches to spatial time series data preprocessing. Presented experiment results validate the approach and show that it performs well in a real-world scenario.