Multimodal crop cover identification using deep learning and remote sensing

Remote sensing is increasingly being used in agriculture and smart farming. Crop cover identification is a major challenge that is useful in the identification of a particular crop at scale. Various studies are conducted to address this challenge using remote sensing and machine learning techniques, but there is still room for improvement in predictive performance. This study has addressed the problem by incorporating multiple modalities for classification modeling. The study has used high-resolution satellite images to perform classification using convolutional neural networks. Densenet201 provided the highest classification accuracy among five candidate architectures. NDVI is calculated from medium-resolution images to be used as a feature that is combined with weather records containing temperature, rainfall, and humidity. A classification ensemble is trained on these features to perform crop classification. Five classification models are trained to select the best classification model. The classification models are evaluated with a train/test split as the models are trained using 60% of the data whereas 20% data is used for validation and 20% for testing. A meta-learner is trained on classification probabilities of both of these classifiers and final class label is obtained. Among the four meta-learners, support vector machines provided best learning and yielded a classification accuracy of 98.83% and an f1-score of 98.78%. High classification performance of the proposed approach indicate that multimodality and meta-learners are useful choices to improve the predictive performance for crop cover identification and can be successfully used for this task.