Visual monitoring of weld penetration in aluminum alloy GTAW based on deep transfer learning enhanced by task-specific pre-training and semi-supervised learning

Appropriate weld penetration is of vital significance for ensuring the welding quality of gas tungsten arc welding (GTAW). Visual monitoring based on deep learning has been widely applied in weld penetration monitoring. However, deep learning requires a large number of labeled samples to achieve satisfactory performance. Deep transfer learning (DTL) is an effective technique to address this issue, but the famous ImageNet dataset may not be suitable for pre-training a deep learning model for weld penetration prediction. In this study, a visual monitoring approach for weld penetration of aluminum alloy GTAW based on DTL enhanced by task-specific pre- training and semi-supervised learning (SSL) is proposed to obtain better prediction accuracy of the backside bead width with limited labeled data. Firstly, an active vision method is used to capture images of the weld pool. Next, a task-specific pre-training method is designed by constructing a keypoint localization task to pre-train a deep learning model with an encoder-decoder architecture, and SSL is introduced to reduce the required number of labeled data in pre-training. Finally, an encoder-based regression model is constructed and fine-tuned to predict the backside bead width. It is found that by using SSL in task-specific pre-training, the keypoint localization model trained with only 40 labeled samples can achieve ideal performance, and the performance of SSL outperforms fully-supervised learning (FSL) in terms of both keypoint localization accuracy and robustness to the randomness of labeled training samples. Moreover, the mean prediction error of backside bead width after finetuning is only 0.176 mm, which is reduced by 29.9 % compared to using ImageNet for pre-training. The proposed method also has good real-time performance and thus has the capability to be applied in the real-time monitoring and control of weld penetration.