Enhanced Aiot Multi-Modal Fusion for Human Activity Recognition in Ambient Assisted Living Environment

MethodologyHuman activity recognition (HAR) has emerged as a fundamental capability in various disciplines, including ambient assisted living, healthcare, human-computer interaction, etc. This study proposes a novel approach for activity recognition by integrating IoT technologies with Artificial Intelligence and Edge Computing. This work presents a fusion HAR approach that combines data readings from wearable sensors such as accelerometer and gyroscope sensors and Images captured by vision-based sensors such as cameras incorporating the capabilities of Long Short-Term Memory (LSTM) and Convolutional Neural Network (CNN) models. The aim of fusing these models is to capture and extract the temporal and spatial information, improving the accuracy and resilience of activity identification systems. The work uses the CNN model to find spatial features from the images that represent the contextual information of the activities and the LSTM model for processing sequential accelerometer and gyroscope sensor data to extract the temporal dynamics from the human activities.ResultsThe performance of our fusion approach is evaluated through different experiments using varying parameters and applies the best-suited parameters for our model. The results demonstrate that the fusion of LSTM and CNN models outperforms standalone models and traditional fusion methods, achieving an accuracy of 98%, which is almost 9% higher than standalone models.ConclusionThe fusion of LSTM and CNN models enables the integration of complementary information from both data sources, leading to improved performance. The computation tasks are performed at the local edge device resulting to enhanced privacy and reduced latency. Our approach greatly impacts real-world applications where accurate and reliable HAR systems are essential for enhancing human-machine interaction and monitoring human activities in various domains.