Generative Semantic Domain Adaptation for Perception in Autonomous Driving

Autonomous driving systems depend on their ability to perceive and understand their environments for navigation. Neural networks are the building blocks of such perception systems, and training these networks requires vast amounts of diverse training data that includes different kinds of driving scenarios in terms of terrains, object categories, and adverse illumination/weather conditions. However, most publicly available traffic datasets suffer from having been sampled under clean weather and illumination conditions. Data augmentation is often used as a strategy to improve the diversity of training data for training machine learning-based perception systems. However, standard augmentation techniques (such as translation and flipping) help neural networks to generalize over simple spatial transformations and more nuanced techniques are required to accurately combat semantic variations in novel test scenarios. We propose a new data augmentation method called “semantic domain adaptation” that relies on the use of attribute-conditioned generative models. We show that such data augmentation improves the generalization capability of deep networks by analyzing their performance in perception-based tasks such as classification and detection on different datasets of traffic objects that are captured (i) at different times of the day and (ii) across different weather conditions, and comparing with models trained using traditional augmentation methods. We further show that GAN-based augmented classification models are more robust against parametric adversarial attacks than the non-GAN-based augmentation models.