Fruit freshness detection based on multi-task convolutional neural network

Background: Fruit freshness detection by computer vision is essential for many agricultural applications, e.g., automatic harvesting and supply chain monitoring. This paper proposes to use the multi-task learning (MTL) paradigm to build a deep convolutional neural work for fruit freshness detection. Results: We design an MTL model that optimizes the freshness detection (T1) and fruit type classification (T2) tasks in parallel. The model uses a shared CNN (convolutional neural network) subnet and two FC (fully connected) task heads. The shared CNN acts as a feature extraction module and feeds the two task heads with common semantic features. Based on an open fruit image dataset, we conducted a comparative study of MTL and single-task learning (STL) paradigms. The STL models use the same CNN subnet with only one specific task head. In the MTL scenario, the T1 and T2 mean accuracies on the test set are 93.24% and 88.66%, respectively. Meanwhile, for STL, the two accuracies are 92.50% and 87.22%. Statistical tests report significant differences between MTL and STL on T1 and T2 test accuracies. We further investigated the extracted feature vectors (semantic embeddings) from the two STL models. The vectors have an averaged 0.7 cosine similarity on the entire dataset, with most values lying in the 0.6-0.8 range. This indicates a between-task correlation and justifies the effectiveness of the proposed MTL approach. Conclusion: This study proves that MTL exploits the mutual correlation between two or more relevant tasks and can maximally share their underlying feature extraction process. we envision this approach to be extended to other domains that involve multiple interconnected tasks.