DESIGN AND IMPLEMENTATION OF A DEEP LEARNING BASED HAND GESTURE RECOGNITION SYSTEM FOR REHABILITATION INTERNET-OF-THINGS (RIOT) ENVIRONMENTS USING MEDIAPIPE

Frequent hospital visits for hand rehabilitation exercises, such as strengthening and opposition exercises, present significant challenges, especially for patients in remote areas. This paper addresses this problem by developing a Rehabilitation Internet-of-Things (RIOT) system that utilizes MediaPipe with its pre-trained Deep Learning (DL) to deliver real-time feedback during hand rehabilitation exercises alongside Web Assembly (WASM) for efficient processing. The system's objective is to provide precise, real-time tracking of hand movements, enabling patients to perform exercises at home by maintaining an optimal distance between the camera and hand placement, ensuring ideal room lighting conditions across IoT devices such as mobile phones' front cameras and webcams, while healthcare professionals remotely monitor their progress. The methodology involves the integration of MediaPipe for detecting hand landmarks and adaptive sensitivity algorithms to ensure reliable recognition across different environments, such as varying lighting and hand positions. Future work could incorporate additional deep-learning models like CNNs and RNNs to enhance gesture classification accuracy. Several limitations, including latency and distance sensitivity, are addressed in this system with edge computing alongside adaptive algorithms. The key contributions of this research are as follows: First, developing a real-time and cost-effective solution for remote stroke rehabilitation. Second, accuracy is improved by integrating MediaPipe with deep learning techniques. Lastly, latency issues and accuracy challenges at extended distances are alleviated by employing innovative calibration methods and adaptive adjustments. Initial trials demonstrate promising results, though further testing is required under real-world conditions to validate the system's effectiveness fully.