Rectified self-supervised monocular depth estimation loss for nighttime and dynamic scenes

Self-supervised monocular depth estimation has attracted much attention in computer vision recently. However, most existing methods assume that the scenes are static and the photometric is consistent, so that their performances tend to degrade significantly in nighttime and dynamic scenes. To address this issue, this paper proposes a self-supervised monocular depth estimation model to tackle two challenges. One is the drastic photometric changes problem due to underexposure of distant areas in nighttime scenes, the other is the moving objects problem in dynamic scenes. In the proposed model, an Effective Area Photometric Loss function (EAPL) is designed which is gated by the Effective Area Mask (EAM) and Potentially Moving Objects Mask (PMOM). Then, a motion flow network is introduced to estimate the motion of moving objects, and a Motion Flow Loss function (MFL) is proposed based on three facts, i.e., the motion flow of static objects should be zero, most moving objects in autonomous driving scenarios are approximately rigid objects, and the relative motion flows between consecutive frames should be mutually inverse. Finally, a decoupled training approach is provided to facilitate the optimization process of the model. Experimental results show that our model achieves state-of-the-art or second best performance on the nuTonomy Scenes (nuScenes) and Dense Depth for Autonomous Driving (DDAD) dataset which contains many nighttime or dynamic scenes, and also achieves competitive performance on the Karlsruhe Institute of Technology and Toyota Technological Institute at Chicago (KITTI) dataset which is dominated by daytime and static scenes. Codes are available at https://github.com/pandaswfas/effdepth.