Automatic image annotation is the process of assigning keywords to digital images depending on the content information. Automatically assigning keywords to images is of great interest as it allows one to index, retrieve, and understand large collections of image data. Many techniques have been proposed for image annotation in the last decade that gives reasonable performance on standard datasets. However, most of these works fail for complex models and requires subsequent training. In this work, we are trying to propose the model which use image as a whole. Features are extracted from the whole image and a relation is explored between the image and the annotation words. Since, we use the whole image; so that the problematic segmentation process is avoided. The hierarchical annotation system proposed in this study performance evaluation annotation systems based on holistic approaches.