Statistically correlated multi-task learning for autonomous driving

Autonomous driving research is an emerging area in the machine learning domain. Most existing methods perform single-task learning, while multi-task learning (MTL) is more efficient due to the leverage of shared information between different tasks. However, MTL is challenging because different tasks may have different significance and varying ranges. In this work, we propose an end-to-end deep learning architecture for statistically correlated MTL using a single input image. Statistical correlation of the tasks is handled by including shared layers in the architecture. Later network separates into different branches to handle the difference in the behavior of each task. Training a multi-task model with varying ranges may converge the objective function only with larger values. To this end, we explore different normalization schemes and empirically observe that the inverse validation-loss weighted scheme has best performed. In addition to estimating steering angle, braking, and acceleration, we also estimate the number of lanes on the left and the right side of the vehicle. To the best of our knowledge, we are the first to propose an end-to-end deep learning architecture to estimate this type of lane information. The proposed approach is evaluated on four publicly available datasets including Comma.ai, Udacity, Berkeley Deep Drive, and Sully Chen. We also propose a synthetic dataset GTA-V for autonomous driving research. Our experiments demonstrate the superior performance of the proposed approach compared to the current state-of-the-art methods. The GTA-V dataset and the lane annotations on the four existing datasets will be made publicly available via