On the Effects of Different Types of Label Noise in Multi-Label Remote Sensing Image Classification

The development of accurate methods for multi-label scene classification (MLC) of remote sensing (RS) images is one of the most important research topics in RS. To address MLC problems, the use of deep neural networks that require a high number of reliable training images annotated by multiple land-cover class labels (multi-labels) has been found popular in RS. However, collecting such annotations is time consuming and costly. A common procedure to obtain annotations at zero labeling cost is to rely on thematic products or crowdsourced labels. As a drawback, these procedures come with the risk of label noise that can distort the learning process of the MLC algorithms. In the literature, most label noise robust methods are designed for single-label classification (SLC) problems in computer vision (CV), where each image is annotated by a single label. Unlike SLC, label noise in MLC can be associated with: 1) subtractive label noise (a land cover class label is not assigned to an image while that class is present in the image); 2) additive label noise (a land cover class label is assigned to an image, although that class is not present in the given image); and 3) mixed label noise (a combination of both). In this article, we investigate three different noise robust CV SLC methods (self-adaptive training (SAT), early-learning regularization, and joint co-regularized training) and adapt them to be robust for multi-label noise scenarios in RS. During experiments, we study the effects of different types of multi-label noise and evaluate the adapted methods rigorously. To this end, we also introduce a synthetic multi-label noise injection strategy that is more adequate to simulate operational scenarios compared to the uniform label noise injection strategy, in which the labels of absent and present classes are flipped at uniform probability. Further, we study the relevance of different evaluation metrics in MLC problems under noisy multi-labels. On the basis of the theoretical and experimental analyses, some guidelines for a proper design of label noise robust MLC methods are derived.