A coarse-to-fine leaf detection approach based on leaf skeleton identification and joint segmentation

Plant leaf detection and segmentation are challenging tasks for in-situ plant image analysis. Here, a novel leaf detection scheme is proposed to detect individual leaves and accurately determine leaf shapes in natural scenes. A leaf skeleton-extraction method was developed by analysing local image features of skeleton pixels. Approximate positions of individual leaves were determined according to the main leaf skeleton. Sub-images containing only single target leaves were extracted from whole plant images according to position and size of the main skeleton. Accurate leaf analysis was conducted on the sub-images of individual leaves. Leaf direction was calculated by examining the structure of the main leaf skeleton. Joint segmentation by combining region and active shape model was presented to accurately elucidate leaf shape. Leaf detection was implemented using deep learning approach, Faster R-CNN. A plant leaf image dataset containing four types of leaf images of different complexity was built to evaluate detection algorithms. Plant leaves with occlusions and complex backgrounds were effectively detected and their shapes accurately determined. Detection accuracy of the proposed method was 81.10%-100%, and 86.75%-100% for Faster R-CNN. The method demonstrated a comparable detection ability to that of Faster R-CNN. Furthermore, the rates of success to determine leaf direction by our method ranged between 89.06% and 100%, while the average measurement difference was 1.29 degrees compared with manual measurement. The accuracy of shape measurement was 75.95%-100% for all types of plant images. Therefore, this method is accurate and stable for precise leaf measurements in agricultural applications. (C) 2021 IAgrE. Published by Elsevier Ltd. All rights reserved.