An Improved Attention and Hybrid Optimization Technique for Visual Question Answering

In Visual Question Answering (VQA), an attention mechanism has a critical role in specifying the different objects present in an image or tells the machine where to focus by invoking the vivid visuals in an image. However, the current VQA systems uses the region-based or bounding box based image features to learn the attention distribution, which are not capable enough to answer the questions related to foreground or background object present in an image. In this paper, we have proposed a VQA model that uses image features which are effective enough to answer questions related to foreground object and background region. Also, we have used graph neural network to encode the relationship between the image regions and objects in an image. Further, we have generated image captions based on these visual relationship based image representation. Thus, the proposed model uses two attention modules to take advantage of each other’s knowledge, to generate more influential attention modules together with the captions based image representation to extract the features which are capable enough to answer questions related to the foreground object and background region. Finally, the performance of proposed architecture is improved by combining the hybrid simulated annealing-Mantaray Foraging Optimization (SA-MRFO) algorithm, which selects the optimal weight parameter for the proposed model. To estimate the performance of the proposed model, two benchmark datasets are used: VQA 2.0 and VQA-CP v2.