Human activity recognition based on multiple inertial sensors through feature-based knowledge distillation paradigm

In recent years, numerous high accuracy methods have been developed for classifying activities using multi inertial sensors. Despite their reliability and precision, they suffer from high computational cost and which make them improper for deploying in edge devices that are limited resources. This paper addresses this drawback by employing a knowledge distillation (KD) paradigm which maps tri-axial multi signals into single axis signals, thus; it can recognize activities with fewer number of signals and consequently less computation. In this method, a big teacher model is trained in advanced with three IMU sensors each of which have tri-axial signals. Then, a small student model is trained with just one of the axes of these sensors under monitoring of teacher which reduces the number of signals. Tucker decomposition is also exploited in order to improve KD performance by separating a core tensor from feature maps that has more informative knowledge. Evaluation of our method on REALDISP dataset demonstrates that the student model could achieve accuracy of 92.90% with much less complexity making it suitable for embedded devices. Moreover, it outperforms in comparison to other state-of-the-art KD approaches.