Integrated use of different content derivation techniques within a multimedia database management system

As amounts of publicly available video data grow, the need to automatically infer semantics from raw video data becomes significant. In this paper, we address the use of three different techniques that support that task, namely, spatio-temporal rule-based method, hidden Markov models, and dynamic Bayesian networks. First, the application of these techniques for detection and recognition of diverse events is briefly described using two case studies (Tennis and Formula 1). We explain the relationships and differences of the three approaches, as well as benefits of their integrated use. Then the focus is moved to the main point of the paper, which is the integration of the aforementioned techniques within a database management system, which provides efficient, flexible, scalable, and domain independent content-based video retrieval. We identify and consider the most important issues when extending a traditional database management system with content-based video retrieval functionality, namely issues concerning video data models, dynamic feature extraction, and extensions of different layers of database architecture. The advantages of the integrated system are demonstrated on examples from our two case studies. (C) 2004 Elsevier Inc. All rights reserved.