Evaluation of deep learning algorithm for crop identification based on GF-6 time series images

Crop type mapping is one of the most important tools with medium and high spatial resolution satellite images in monitoring services of modern agriculture. Taking Heilongjiang Province of northeast China as a study area, this study aims to evaluate the state-of-the-art deep learning in crop type classification. A comparison was made on the Convolution Neural Network (CNN), Recurrent Neural Network (RNN), and Attention Mechanism (AM) for the application in crop type classification, while the traditional random forest (RF) model was also used as the control. Six models of deep learning were Temporal Convolutional Neural Network (TempCNN), Multi Scale 1D Residual Network (MSResNet), Omniscale Convolutional Neural Network (OmniscaleCNN), Long Short-Term Memory (LSTM), STAR Recurrent Neural Network (StarRNN), and Transformer. The specific procedure was as follows. First, GF-6 wide-field view image time series was acquired between April and November in the Lindian and Hailun study area, northeast of China, in order to extract the features of three types of crops at different growth stages. The resulting image time series used in the Lindian and the Hailun was composed of 41 and 48 GF-6 images, respectively. The preprocessing workflow included RPC correction, radiometric calibration, convert to top-of-atmospheric and surface reflectance using 6S atmospheric correction. The image interpolation and global min-max normalization were also applied to fill the empty pixel, further improving the convergence speed and stability of neural networks. The ground truth data was manually labelled using a field survey combined with GF-2 high-resolution image to generate datasets for train and evaluation. The datasets included six crops, such as rice, maize, soybean, water, urban and rest, covering 2 003 629 pixels in Lindian, 935 679 pixels in Hailun. Second, all models were trained and evaluated in Lindian, according to the differences between CNN, RNN, AM, and RF. All models achieved an overall accuracy of 93%-95%, and F1-score above 89%, 84%, and 97% for soybean, maize, and rice, respectively, where three major crops were from both study areas. Thirdly, the trained model in Lindian was transferred to that in Hailun, where the overall classification accuracy of each model declined between 7.2% to 41.0%, due to the differences of land cover classes and temporal composition of the data. Among CNNs, the accuracy of MSResNet barely changed to recognize three types of crops after transfer. Since OmniScaleCNN was automatically adjusted the size of the convolution filter, the accuracy of OmniScaleCNN after the transfer was better than that of TempCNN. A forget gate was utilized in the LSTM and StarRNN among RNNs, in order to avoid gradient explosion and disappearance in the classification, where the overall accuracy declined less than 10% after transfer. However, the accuracy of attention-based Transformer and RF dropped significantly. All models performed better on the distribution of three types of crops under the condition that the spatial location and temporal composition of data remain unchanged, in terms of visual analysis of classified images. Two CNN or RNN models were expected to accurately identify the general distribution of all land cover classes, under the varying spatial location and temporal composition. Furthermore, the run time of each deep learning was within 1 h, less than 6.2 times of random forest. Time consumption in the whole process was associated with the model training, as well as the image treatment for the Hailun study area covering an area of about 10 000 km². Correspondingly, deep learning presented a high-precision and large-scale crop mapping, in terms of classification accuracy and operating efficiency, particularly that the transfer learning performed better than before. © 2021, Editorial Department of the Transactions of the Chinese Society of Agricultural Engineering. All right reserved.