3D pose estimation and localization of construction equipment from single camera images by virtual model integration

Computer vision technologies are receiving much attention for automatic site monitoring. However, previous approaches mainly focused on two-dimensional (2D) pixel information, and there was a limit to providing detailed three-dimensional (3D) pose information. Some studies have built a training database only using virtual models, but there were still limitations in representing the actual equipment. To address these limitations, the authors propose a method for estimating 3D poses of equipment from single camera images by integrating virtual models. The proposed method consists of three main processes: (1) 2D - 3D annotation using a virtual model; (2) preprocessing for pose estimation in two ways (i.e., image matching or key points detection); and (3) 3D pose localization. As a result, the root mean square error for the 3D pose of construction equipment was 1.49 m (using image matching) and 1.56 m (using key points detection). The results imply that the proposed method successfully estimated the 3D pose of the equipment with a single camera. The findings of this study can contribute to detailed productivity or safety analysis based on more diverse 3D information than 2D pixel-based analysis.