Robust feature matching via progressive smoothness consensus

Feature matching is a long-standing fundamental and critical problem in computer vision and photogrammetry. The indirect matching strategy has become a popular choice because of its high precision and generality, but it finds only a limited number of correct matches, and the mismatch removal phase does not utilize the critical feature descriptors. To this end, this paper proposes a novel and effective feature matching method, named Progressive Smoothness Consensus (PSC). Our PSC designs an objective function to directly construct correct matches from two feature point sets. To optimize the objective, we introduce a stepwise strategy, where a small but reliable match set with the smooth function is used as initialization, and then the correct match set is iteratively enlarged and optimized by match expansion and smooth function estimation, respectively. In addition, the local geometric constraint is added to the compact representation with a Fourier basis, thus improving the estimation precision. We perform the match expansion as a Bayesian formulation to exploit both the spatial distribution and feature description information, thus finding feasible matches to expand the match set. Extensive experiments on feature matching, homography & fundamental matrix estimation, and image registration are conducted, which demonstrate the advantages of our PSC against state-of-the-art methods in terms of generality and effectiveness. Our code is publicly available at https://github.com/XiaYifan1999/Robust-feature-matching-via-Progressive-Smoothness-Consensus.