A Statistical Approach to Hyperparameter Tuning of Deep Learning for Construction Machine Classification

Deep learning methods have shown significant potential in computer vision applications within the field of civil engineering. One important area of research is the detection and classification of construction machines using convolutional neural networks (CNNs). However, a major challenge in adopting machine learning models for vision systems lies in hyperparameter optimization. Therefore, the objective of this paper is to propose a meticulous statistical approach to hyperparameter tuning for deep learning in the visual classification of construction machines. It is noteworthy that the method for selecting hyperparameters utilizes statistical concepts, including analysis of variance and the Tukey test. Moreover, three research questions were formulated to address statistical differences in performance when varying the CNN architecture, optimizer and configuration (architecture and optimizer). Two additional research questions were posed to determine the recommended hyperparameters for the task and assess performance differences for non-selected hyperparameters. For this purpose, 18 hyperparameter combinations were analyzed, encompassing three CNN architectures (DenseNet, Mobilenet and VGG16) and six optimizers (adadelta, adagrad, rmsprop, adam, adamax and sgd). The Alberta Construction Image Dataset, consisting of 2850 images of three construction machines (excavator, dump truck and concrete mixer truck), was used in the experiments. The results demonstrate that the hyperparameter configurations exhibit statistically significant differences in image classification. Furthermore, the recommended combination (DenseNet ×\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\times $$\end{document} adagrad) yielded the highest average accuracy results in both binary tests (90.0%) and multiclass classification (77.8%).