Evaluation of Local Descriptors for Automatic Image Annotation

Feature extraction is the first and often also the crucial step in many computer vision applications. In this paper we aim at evaluation of three local descriptors for the automatic image annotation (AIA) task. We utilize local binary patterns (LBP), patterns of oriented edge magnitudes (POEM) and local derivative patterns (LDP). These descriptors are successfully used in many other domains such as face recognition. However, the utilization of them in the AIA field is rather infrequent. The annotation algorithm is based on the K-nearest neighbours (KNN) classifier where labels from K most similar images are "transferred" to the annotated one. We propose a label transfer method that assigns variable number of labels to each image. It is compared with an existing approach using constant number of labels. The proposed method is evaluated on three image datasets: Li photography, IAPR-TC12 and ESP. We show that the results of the utilized local descriptors are comparable to, and in many cases outperform the texture features usually used in AIA. We also show that the proposed label transfer method increases the overall system performance.