Learning niche features to improve image-based species identification

Species identification is a critical task of ecological research. Having accurate and intelligent methods of species identification would improve our ability to study and conserve biodiversity with saving much time and effort. But image-based deep learning methods have rarely taken account of domain knowledge, and perform poorly on imbalanced training dataset and similar species. Here, we propose NicheNet which combines ecological niche model and image-based deep learning model together expanding from the joint model framework invented by previous research. We incorporate an optimization process for NicheNet, and examine its performance on identifying Chinese Galliformes comparing with image-only model and Geo-prior-based model. For assessing the ability of both NicheNet and image-only models on distinguishing similar species, we initiate a new criterion named Near Error Rate in this study which decomposes identification error rate on each species along its classification. We show that the joint models gain significantly improvement on identifying our study species compared with image-only model. Against on an image-only baseline 82.5%, we observe 7.73% improvement in top-1 accuracy for NicheNet, and a 6.21% increase for Geo-prior-based model. NicheNet gains a 15.3% increment of average F1-Score and gets a -8.5% mean decrement in Near Species Error Rate while-4.4% in Near Genus Error Rate. Further cases analysis shows the background mechanism of NicheNet to improve image-only models. We demonstrate that NicheNet can learn the features from images and niche-prior, which is generated by niche model and finer than geo-prior by Geo-prior-based model, together to promote its ability on identifying species, especially on those with small training dataset and similar outlook. It is a flexible model framework, and we can introduce more biological features and domain models to strengthen its accuracy and robust in the future work. Our study shows that NicheNet can be widely adopted and will accurately accelerate automatic species identification task for biodiversity research and conservation.