Comparison analysis on land cover area estimators: confusion matrix calibration and regression

Remote sensing can provide timely, economic and objective data with a large scope, and thus has become the main data source for land cover/use area estimation. However, the classification result cannot be directly used to calculate the area of a given land cover type because of the commission and omission errors. Confusion matrix calibration (direct estimator and inverse estimator) and regression (ratio estimator and simple regression estimator) are two commonly used approaches by which one can estimate the accurate area of a given land cover type from a classification map. However, their applied conditions and relative efficiency are unclear. The two approaches (including four specific estimators) were compared with a simple expansion estimator, which is regarded as the basic estimator in a comparison process, by using the same simulated images under a unified sampling scheme. The main purpose is to explore the accuracy and stability of each estimator in different overall classification accuracy and sampling ratios. The simulated images were produced by adding different amounts of errors into the classification map of real remote sensing data. These errors fall into two categories. One is the errors within patches, which result from the similarity of spectral information. The other is the errors located around the boundary of different land cover types, because of the misclassification of mixed pixels. According to the distribution rule of remote sensing classification error and heterogeneity, we calculated the error possibility for each pixel, and then altered the class label of error pixels. There were four different images we simulated by a procedure of IDL 7.0 to represent various overall accuracy levels of classification in practical cases. Then, two evaluation criteria were adopted, with average absolute relative bias, indicating the accuracy of estimation, and coefficient of variance, representing the stability of estimation. The results suggest that: (1) Confusion matrix calibration and regression can provide more accurate and stable estimates than simple expansion estimator does, which means that the simulated images, also the remote sensing classification map in practical cases, play an positive role in estimation, especially in improving the stability degree. (2) In the experiments with different overall classification accuracy and sampling ratio, the two evaluation criteria value of simple regression estimator and ratio estimator are extremely close, and they all show higher accuracy and stability compared with confusion matrix calibration methods, which indicates that regression methods have more advantages to some degree. In addition, confusion matrix calibration methods are more sensitive to classification accuracy than that of regression methods. The reason is that confusion matrix calibration methods are interested in not only the amount of pixels which are classified correctly, but also the land cover type which one error pixel falls into. Regression methods use the information from sampling units more comprehensively, and the relationship between samples and auxiliary data stands for the general connection between ground truth and remote sensing classification, and also reduces the influence of an individual sample. (3) When the overall accuracy of classification is low (≤60%), estimates from the inverse estimator are similar to or even worse than the simple expansion estimator. Therefore, an inverse estimator should not be used under this condition.