Enhancing the reliability of image classification using the intrinsic features

Usually, the features extracted by deep learning models include both the intrinsic features and the related features, where the features from the category-related object itself are called the intrinsic features and the features from the background are called the related features. We argue that the classification relying on the intrinsic features is reliable, so the reliability of classification can be defined as the dependency on the intrinsic features. However, utilizing the intrinsic features in image classification has not been extensively studied. One reason is that the widely used classification accuracy cannot directly represent the reliability. Thus, we propose a novel metric called Reliable Classification Ratio (RCR) to estimate the reliability of the classification results, and as a complementary metric to the classification accuracy. Utilizing RCR and the accuracy, it is found that the classification results of the deep learning models heavily rely on the related features, which demonstrates that the results are unreliable. To deal with the problem, we propose a contrastive training method that regards the background as an extra category. The image that includes the category-related object and the image that excludes the category-related object are both used for feature learning, which guides deep learning models to make better use of the intrinsic features in the classification. Experiments prove that the proposed method significantly improves the reliability while maintaining classification accuracy.(c) 2023 Elsevier B.V. All rights reserved.