UAV-based Inspection of Airplane Exterior Screws with Computer Vision

We propose a new approach to detect and inspect aircraft exterior screws. An Unmanned Aerial Vehicle (UAV) locating itself in the aircraft frame thanks to lidar technology is able to acquire precise images coming with useful metadata. We use a method based on a convolutional neural network (CNN) to characterize zones of interest (ZOI) and to extract screws from images; methods are proposed to create prior model for matching. Classic matching approaches are used to match the screws from this model with the detected ones, to increase screw recognition accuracy and detect missing screws, giving the system a new ability. Computer vision algorithms are then applied to evaluate the state of each visible screw, and detect missing and loose ones.