Automatic Annotation of Multimedia Content by User Clickthroughs: Enhancing the Performance of Multimedia Search Engines

Content-based multimedia retrieval is a very hot research topic, applicable to several domains. Traditional feature vector based retrieval methods cannot provide semantically meaningful results. Additionally manual annotation is both time/money consuming and user-dependent. To address these problems in this paper we present an approach to automatically annotate multimedia files by incorporating clickthrough data of search engines. In particular the query-log of the search engine in connection with the log of links the users clicked on in the presented ranking, are analyzed in order to assign keywords to selected content. A query extension method is also proposed in order to agitate the pool of files and bring content with similar visual features to the Surface. This is very important since users typically select only the first files of the ranking by clicking on them. The proposed method is feasible even for large sets of queries and features and theoretical results are verified in a controlled experiment, which shows that the method can effectively annotate multimedia files and significantly enhance the performance Of Multimedia search engines.