Performance evaluation of large-scale object recognition system using bag-of-visual words model

Object recognition technology is usually used for recognizing specific objects, such as book covers, landmarks, vehicles, etc. This technology is supported by multi-dimensional local image descriptors in most situations. These descriptors are designed to be robust to the environmental changes, such as illumination change, view angle change, scale change, etc. If there are many target objects in your database, object recognition using large scale local image descriptor database may not be a trivial task, because of the high dimensionality of the local image descriptors. For consistent responses from a large-scale database with a reasonable time delay, we need to have a proper data structure which supports the indexing and querying functionality. A vocabulary tree is a data structure based on local image descriptors, and this data structure is commonly used to cope with massive databases containing local image descriptors. By using a vocabulary tree, a local image descriptor can be mapped to a vocabulary tree's leaf node ID, constructing a visual word for object recognition. Visual words are then effectively exploited by a traditional text retrieval engine. In this study, we built a large-scale object recognition system using a vocabulary tree that had leaf nodes of 1 million Scale-Invariant Feature Transform (SIFT) descriptors, which is the most promising local image descriptor in terms of precision. We implement proposed system using publicly available software so that further enhancements and/or reproducibility would be easily accomplished. We then compared and evaluated the proposed system's performance with the current MPEG CDVS (Compact Descriptors for Visual Search) standard using a database containing two dimensional planar object datasets of three categories with one million distracter images. In addition to these datasets, which are equivalent to those of CDVS, we add a new dataset which are made to mimic realistic occlusion and clutter effects. Experimental results show that our proposed system's performance is comparable to that of the CDVS achieving 90 % precision at 5 s retrieval time. We also find characteristics of vocabulary tree limiting adaptation to a specific application domain.