Thematic video indexing to support video database retrieval and query processing

This paper presents a novel video database system, which caters for complex and long videos, such as documentaries, educational videos, etc. As compared to relatively structured format videos like CNN news or commercial advertisements, this database system has the capacity to work with long and unstructured videos. A video's metadata is stored in a database. This metadata is used for indexing and querying the video. In this model, metadata contains information about segments (combinations of frames) and scenes (collections of segments that represent the same content). The metadata is organized in such a way that it can be used to navigate a theme (concept or idea). Annotations describing scenes are linked in hierarchical manner to create a story line in the video, where thematic indexing is used to develop a video catalogue. Thematic indexing is a novel way to track a story in a video. A video contains many themes, which are implicitly related to each other. In order to resolve queries about particular scenes in a video, the scenes are stored hierarchically, which provides "is-a" or "have-part" relations between them. We have tested our model on one of the hour-long documentaries made for television by the Earl Mountbatten of Burma. The results show that a user can easily query for natural scenes or events such as Japanese Soldiers landing in Burma in World War 2, Allied forces defending the border, air bombings, soldiers marching in fields etc. The database system has been designed to ensure MPEG-4 compatibility, The metadata and other information is kept in such a format that it could be easily converted as captions to Audio Video Objects (AVOs) at the access unit layer in the MPEG-4 System Layer Model.