Assessment of the handcart pushing and pulling safety by using deep learning 3D pose estimation and IoT force sensors

Pushing and pulling (P&P) are common and repetitive tasks in industry, which non-ergonomic execution is among major causes of musculoskeletal disorders (MSD). The current safety management of P&P assumes restrictions of maximal weight, distance, height - while variable individual parameters (such as the P&P pose ergonomic) remain difficult to account for with the standardized guides. Since manual detection of unsafe P&P acts is subjective and inefficient, the aim of this study was to utilize IoT force sensors and IP cameras to detect unsafe P&P acts timely and objectively. Briefly, after the IoT module detects moments with increased P&P forces, the assessment of pose ergonomics was performed from the employee pose reconstructed with the VIBE algorithm. The experiments showed that turn-points correspond to the high torsion of torso, and that in such moments poses are commonly non ergonomic (although P&P forces are below values defined as critical in previous studies - their momentum cause serious load on the human body). Moreover, the analysis revealed that the loading/unloading of a cargo are also moments of frequent unsafe P&P acts - although they are commonly neglected when studying P&P. The experimental validation of the solution showed good agreement with motion sensors and high potential for monitoring and improving P&P workplace safety. Accordingly, future research will be directed towards: 1) acquisition of P&P data sets for direct recognition and classification of unsafe P&P acts; 2) incorporation of wearable sensors (EMG and EEG) for detecting fatigue and decrease of physical abilities.