Automatic vision-based calculation of excavator earthmoving productivity using zero-shot learning activity recognition

Recently, vision-based methods have been widely used to analyze the construction productivity based on onsite videos owing to their low cost, simple deployment, and easy maintenance. However, existing vision-based methods rely on supervised learning for activity recognition, which is computationally intensive owing to the necessity of labeling large-scale training datasets. To address this problem, this paper describes a vision-based method for automatically analyzing excavators' productivities in earthmoving tasks by adopting zero-shot learning for activity recognition. The proposed method can identify activities of general construction machines (e.g., excavators and loaders) without pre-training or fine-tuning. To verify the feasibility, the proposed method has been tested on videos recorded from real construction sites. The accuracy values for activity recognition and productivity evaluation are 86% and 87.8%, respectively.