Multimodal Convolutional Neural Network Model for Protective Behavior Detection based on Body Movement Data

Chronic pain treatment is a significant challenge in the healthcare industry. Physiotherapists tailor physical activity to a patient's activity based on their expression in protective behavior through pain recognition and find the special equipment to help them perform the necessary tasks. The technology can detect and assess pain behavior that could support the delivery of personalized therapies in the long-term and self-directed management of the condition to improve engagement in valued everyday activities. In this paper, we present an approach for task 1 of the Affective Movement Recognition (AffectMove) Challenge in 2021. Our proposed approach using deep learning helps detect persistent protective behavior present or absent during exercise in a person with chronic pain, based on the full-body joint position and back muscle activity of EmoPain challenge 2021 dataset. We employ convolutional neural networks by stacking residual blocks for the multimodal model. Moreover, we suggest new feature groups as additional inputs that help to increase performance for protective behavior. The proposed approach achieves an F1 score of 78.56% on validation set and 59.11% on test set. The proposed approach also outperforms previous baselines in detecting protective behavior from the EmoPain dataset.