Semantics supervised cluster-based index for video databases

High-dimensional index is one of the most challenging tasks for content-based video retrieval (CBVR). Typically, in video database, there exist two kinds of clues for query: visual features and semantic classes. In this paper, we modeled the relationship between semantic classes and visual feature distributions of data set with the Gaussian mixture model (GMM), and proposed a semantics supervised cluster based index approach (briefly as SSCI) to integrate the advantages of both semantic classes and visual features. The entire data set is divided hierarchically by a modified clustering technique into many clusters until the objects within a cluster are not only close in the visual feature space but also within the same semantic class, and then an index entry including semantic clue and visual feature clue is built for each cluster. Especially, the visual feature vectors in a cluster are organized adjacently in disk. So the SSCI-based nearest-neighbor (NN) search can be divided into two phases: the first phase computes the distances between the query example and each cluster index and returns the clusters with the smallest distance, here namely candidate clusters; then the second phase retrieves the original feature vectors within the candidate clusters to gain the approximate nearest neighbors. Our experiments showed that for approximate searching the SSCI-based approach was faster than VA(+)-based approach; moreover, the quality of the result set was better than that of the sequential search in terms of semantics.