Using Kinect as sensor for smart spraying

This work is aimed to solve the problem of fruit tree canopy characterization for spraying purposes. This proposal is based on using a depth map and a RGB image provided by the vision sensor "Kinect" from Microsoft to perform smart spraying. Through the depth map the density of trees can be estimated and thereby it is possible to determine which nozzles should be turned on/off at every moment. Furthermore, algorithms to apply pesticides only to leaves and/or fruits as desired were created. The software was developed in Matlab that allows the acquisition of the depth map and RGB image from the "Kinect" sensor. This software communicates with the "Kinect Windows SDK", processes the information, and then provides information regarding the presence and location of leaves and/or fruit. To identify leaves, classification and identification algorithms were applied. The classification algorithms used were "Fuzzy C-Means with Gustafson Kessel" FCM-GK and "K-Means". In this framework the centroids generated by FCM are used as seed for K-means, in order to accelerate the implementation and maintain temporal consistency in the groups generated by the K-means algorithm. Classification algorithms were applied over the images transformed to the L*a*b* color spaces, specifically over the channels a*b* (the chromatics channels) in order to reduce the light effect over the colors. Classification algorithms were tuned to search for four clusters: leaves, porosity, fruits, and trunk. Once the classifier generates the cluster prototype, leaves are identified by using a binary Support Vector Machine that uses as kernel a Gaussian radial basis function. The combinations of all these algorithms have shown low misclassification, yield 4% error on the leaves identification. Besides, these algorithms process up to 8.4 frames per second, allowing its application in real time. Result shows the feasibility of using the "Kinect" sensor to determine where and when to apply pesticide. On the other hand, it also shows the limitation imposed by the lightening conditions. In other words, it is possible to use "Kinect" outdoors, but working during cloudy days, early in the morning or at night using artificial illumination, or adding a sun shield for strong light conditions.