No-Reference Image Quality Assessment for Multiple Distortions Using Saliency Map Based on Dual-Convolutional Neural Networks

No-reference image quality assessment (NR-IQA) is a challenging topic in image processing and image assessment, especially for multiple distortions. We propose a novel NR-IQA method based on a dual-convolutional neural network structure. Both gray and color features in Hue Saturation Value (HSV) space are extracted to train a model in order to improve the accuracy of logistic regression. In addition, our method combines with saliency detection which is in line with human visual system (HVS). The saliency map contains more information of an image and is considered to be the most attractive region. Thus, a well-trained model is used to detect the saliency map of an input image. The saliency map is employed as a weighting function to reflect the important distortion brought by the local region. The first CNN structure includes two convolutional layers with maximum pooling, inception module and two fully connected layers. The second CNN structure only contains convolutional and pooling layers. Our proposed algorithm achieves a Pearson (Linear) Correlation Coefficient (LCC) of 0.964 and a Spearman's Rank Ordered Correlation Coefficient (SROCC) of 0.962 on the LIVE database, which demonstrates the excellent generalization ability in cross database experiment. Furthermore, we fine-tune a CNN model trained on LIVE database to predict multiply distorted image quality. Experiment result shows that our method outperforms the state-of-the-art methods on LIVE multiply distorted image database.