Towards on-site hazards identification of improper use of personal protective equipment using deep learning-based geometric relationships and hierarchical scene graph

Construction sites are one of the most perilous environments where many potential hazards may occur. Personal Protective Equipment (PPE) is an important safety measure used to protect construction workers from accidents. However, PPE usage is not strictly enforced among workers due to all kinds of reasons. This paper proposes a unified model, which enjoys both perceptual and reasoning capabilities, to help to facilitate the safety monitoring work of construction workers to ensure PPE is appropriately used. In contrast to commonly used object detection-based identification approaches, this paper provides a novel solution to identify improper use of PPE by the combination of deep learning-based object detection and individual detection using geometric relationships analysis. Moreover, this paper presents a hierarchical scene graph structure that enables the conditional reasoning for automated hazards identification to address different requirements in each zone of construction sites. The experimental results demonstrate that the proposed approach was capable of identifying the hazards of improper use of PPE with high precision (94.47%) and recall rate (83.20%) while ensuring real-time performance (15.62 FPS on average).