Object-based MPEG-2 video indexing and retrieval in a collaborative environment

In this paper, an object-based video retrieval methodology for search in large, heterogeneous video collections is presented. The proposed approach employs a real-time, compressed-domain, unsupervised algorithm for the segmentation of image sequences to spatiotemporal objects. For the resulting objects, MPEG-7 compliant low-level descriptors describing their color, shape, position and motion characteristics are extracted. These are automatically associated using a fuzzy C-means algorithm with appropriate intermediate-level descriptors, which are part of a simple vocabulary termed object ontology. Combined with a relevance feedback mechanism, this scheme allows the qualitative definition of the high-level concepts the user queries for (semantic objects, each represented by a keyword) and relations between them, facilitating the retrieval of relevant video segments. Furthermore, it allows the collaborative construction of a knowledge base by accumulating the information contributed to the system during feedback by different users. Thus, it enables faster and more accurate retrieval of commonly requested keywords or semantic objects. Experimental results in the context of a collaborative environment demonstrate the efficiency of the proposed video indexing and retrieval scheme.