Generation of synthetic dataset to improve deep learning models for pavement distress assessment

Analysis of automated pavement image distress data requires large annotated datasets that can be used for training deep learning (DL) models. The lack of labeled data is a significant challenge in utilizing DL models for distress detection. Furthermore, developing the ground-truth is difficult in low-data regimes and the inconsistency in ground-truth is a challenge for deploying DL models for analyzing pavement distress data. Consequently, this study proposes the conditional Style-based Generative Adversarial Network (c-StyleGAN2) with Adaptive Discriminator Augmentation (ADA) to create synthetic images of pavement distresses using distress classes as conditional input. A novel technique has been utilized in this study to create high-resolution synthetic pavement images using moderate computational resources by upsampling through a super-resolution model Real-ESRGAN and blended into a no-distress background using Deep Image Blending. Thereafter, to evaluate the usefulness of including synthetic data in pavement condition evaluation, DeepLab v3 + was implemented for semantic segmentation and Mask R-CNN was implemented for instance segmentation on a few scenarios involving aggregated synthetic and real data as well as only real data. The models were trained both with and without using Transfer Learning. The results of this study show that synthetic data aggregation led to an increase in detection accuracy and that utilizing transfer learning by pretraining on synthetic data and fine-tuning on real data yielded the best performance.