CHARM-Deep: Continuous Human Activity Recognition Model Based on Deep Neural Network Using IMU Sensors of Smartwatch

In the present paper, an attempt was made to achieve high-performance continuous human activity recognition (CHAR) using deep neural networks. The present study focuses on recognizing different activities in a continuous stream, which means 'back-to-back' consecutive set of activities, from only inertial measurement unit (IMU) sensors mounted on smartwatches. For that purpose, a new dataset called 'CHAR-SW', which includes numerous streams of daily activities, was collected using smartwatches, and feature representations and network architectures were designed. Experimental comparisons using our own dataset and public datasets (Aruba and Tulum) have been performed. They demonstrated that cascading bidirectional long short-term memory (Bi-LSTM) with featured data performed well in offline mode from the viewpoints of accuracy, computational time, and storage space required. The input to the Bi-LSTM is a descriptor which composed of a stream of the following features: autocorrelation, median, entropy, and instantaneous frequency. Additionally, a novel technique to operate the CHAR system online was introduced and shown to be effective. Experimental results can be summarized as: the offline CHARM-Deep enhanced the accuracy compared with using raw data or the existing approaches, and it reduced the processing time by 86% at least relative to the time consumed in executing the Bi-LSTM classifier directly on the raw data. It also reduced storage space by approximately 97.77% compared with using raw data. The online evaluation shows that it can recognize activities in real-time with an accuracy of 91%.