A new stereo matching energy model based on image local features

This paper constructs an energy model based on local features used in stereo matching. The local features include the similarity between different image areas, the matching cost function pattern, the connection between neighbor pixels, and the occlusion geometric relationship. Based on these features, we define the weight of each data term and smoothing term in the energy function and then design an algorithm to solve the energy model and get disparity results. The significant improvements of this paper include as following. 1) We modify the structure of the energy function. First, we define the weight of the data term based on the reliability of its corresponding disparity result, which is obtained by cost function features and the occlusion geometric relationship. Then we define the weight of the smoothing term by analyzing the characteristic relation between neighbor super-pixels. We can also reduce the computational complexity by detecting and reducing some low-strength connections. 2) We proposed an algorithm based on pairwise Markov random field (MRF) (Taniai et al., IEEE Trans Pattern Anal Machine Intell 40(11): 2725–2739, 2017) and local greedy iteratively, which can be used to solve the energy model. 3) In post-optimation, we select some areas with severe occlusion and fewer matching clues for post-interpolation fitting to optimize the results. The experiment shows that the proposed method reduced the average percentage of bad pixels (in bad 3) to 6.06 on the Middlebury dataset and 1.42 on the KITTI dataset. Finally, we compare our results with those of MC-Cnn (Zbontar and LeCun 2015), CF-Net (Shen et al., 2021), Guided-Stereo (Poggi et al., 2019), Gwc-Net (Guo et al., 2019) and Patchmatch-Net(PM-Net) (Wang et al., 2021) to verify the improved speed and accuracy of our algorithm, especially at recognizing the depth of changing edges and small objects. This paper’s relevant research can contribute to practical engineering practices such as assisted vision, intelligent driving, and robot grasping control.