Bag of words for semantic automatic medical image annotation

A medical report contains many elements such as medical images accompanied by text descriptions. We present in this paper a new approach for semantic automatic annotation of medical images. The proposed approach uses the bag of words model to represent the visual content of the medical image combined with text descriptors based on term frequency–inverse document frequency technique and reduced by latent semantic to extract the co-occurrence between text and visual terms. In a first phase, we are interested in indexing texts and extracting all relevant terms using a thesaurus containing medical subject headings and concepts. In a second phase, medical images are indexed while recovering areas of interest which are invariant to change in scale such as light and tilt. To annotate a new medical image, we use the bag of words model to recover the feature vector. Indeed, we use the vector space model to retrieve similar medical images from the training database. The computation of the relevance value of an image according to a query image is based on the cosine function. To evaluate the performance of our proposed approach, we present an experiment carried out on five types of radiological imaging. The results showed that our approach works efficiently, especially with more images taken from the radiology of the skull. © Springer-Verlag Wien 2014.