Coding Images with Local Features

We develop a qualitative measure for the completeness and complementarity of sets of local features in terms of covering relevant image information. The idea is to interpret feature detection and description as image coding, and relate it to classical coding schemes like JPEG. Given an image, we derive a feature density from a set of local features, and measure its distance to an entropy density computed from the power spectrum of local image patches over scale. Our measure is meant to be complementary to existing ones: After task usefulness of a set of detectors has been determined regarding robustness and sparseness of the features, the scheme can be used for comparing their completeness and assessing effects of combining multiple detectors. The approach has several advantages over a simple comparison of image coverage: It favors response on structured image parts, penalizes features in purely homogeneous areas, and accounts for features appearing at the same location on different scales. Combinations of complementary features tend to converge towards the entropy, while an increased amount of random features does not. We analyse the complementarity of popular feature detectors over different image categories and investigate the completeness of combinations. The derived entropy distribution leads to a new scale and rotation invariant window detector, which uses a fractal image model to take pixel correlations into account. The results of our empirical investigations reflect the theoretical concepts of the detectors.