FoveaMask: A fast and accurate deep learning model for green fruit instance segmentation

In the process of agricultural automation production, efficient and accurate segmentation of target fruit is the basis and guarantee for numerous applications including crop growth monitoring, yield prognosis, and machine picking. Green fruit is apt to affect by complicated scenes such as occlusions and overlaps, as well as the homochromatic background, which brings great challenges to the recognition and segmentation. In this paper, we propose a novel methodology named FoveaMask for improving the robustness and generalization of green fruit segmentation framework. Features of input images are firstly extracted by ResNet and fused by FPN. The classification and bounding-box regression of each spatial position on the feature maps are then carried out directly by the way of full convolution. RoI Align layer is applied to fix feature region of proposals to the same size yet preserve exact spatial locations. Finally, instance-level fruit segmentation is realized by pixel-level classification on each proposal using embedded mask branches. The whole network architecture of the new model does not involve the related design and operation of anchor, which greatly improves the generalization ability for different shape fruits, alleviates the computing and storage resources, and balances the contradiction between accuracy and efficiency simultaneously. Additionally, a position attention module(PAM) is introduced into the embedding mask branch to aggregate the effective information pixels, which can improve the robustness of the segmentation model in the actual complex environment of the orchard. We test the model on green apple and immature persimmon data sets and the experimental results show that FoveaMask performs best in terms of both recognition accuracy and model complexity compared with other 11 different types of detection and segmentation models.