Cultivar discrimination of litchi fruit images using deep learning

Litchi (Litchi chinensis Sonn.) originated from China and many of its cultivars have been produced in China so far during the long history of cultivation. One problem in litchi production and research is the worldwide confusion regarding litchi cultivar nomenclature. Because litchi cultivars can be described in terms of cultivar-dependent fruit appearance, it should be possible to discriminate cultivars of postharvest fruits. In this study, we explored this possibility using recently developed deep learning technology for four common Taiwanese cultivars 'Gui Wei', 'Hei Ye', 'No Mai Tsz', and 'Yu Her Pau'. First, we quantitatively evaluated litchi fruit shapes using elliptic Fourier descriptors and characterized the relationship between cultivars and fruit shapes. Results suggest that 'Yu Her Pau' can be clearly discriminated from others mainly based on its higher length-to-diameter ratio. We then fine-tuned a pre-trained VGG16 to construct a cultivar discrimination model. Relatively few images were sufficient to train the model to classify fruit images with 98.33% accuracy. We evaluated our model using images of fruits collected in different seasons and locations and found the model could identify 'Yu Her Pau' fruits with 100% accuracy and 'Hei Ye' fruits with 84% accuracy. A Grad-CAM visualization reveals that this model uses different cultivar-dependent regions for cultivar recognition. Overall, this study suggests that deep learning can be used to discriminate litchi cultivars from images of the fruit.