Impurity detection of juglans using deep learning and machine vision

Impurity detection is crucial for quantitative image analysis in numerous food safety control and quality inspection applications. Rapid impurity detection of juglans using machine vision still faces challenge due to the complex shapes and color of foreign objects in different postures. Some traditional detection methods require expertly designed constraints and manual model parameters, and they have the poor detection performance and high model maintenance costs. In recent years, deep learning has become a focus in different research fields, because methods based on deep learning are able to directly learn features from training data. In this study, we originally proposed the two-stage convolutional networks to finish the image segmentation and detection of impurities in juglans images in real-time. The proposed segmentation method based on multiscale residual fully convolutional networks and classification method based on convolutional networks automatically can segment images and detect different-sized impurities (e.g., leaf debris, paper scraps, plastic scraps and metal parts) at the same time. The proposed deep-learning method is simpler and more effective, because it avoids extracting features manually, and it not only overcomes the conglomeration phenomenon between juglans and foreign objects in inline images, but also adapts to the disturb from surface abrasion damage on the white transmission belt to avoid error detection in the real factory environment. The proposed method is able to correctly segment 99.4% of the object regions in the test images and to correctly classify 96.5% of the foreign objects in the validation images and correctly detect 100.0% of test images. The segmentation and detection processing time of each image was less than 60 ms. Future work will focus on deep learning using multi-wave imaging and the sorting mechanical control.