Differential Morphological Profile Neural Network for Maneuverability Hazard Detection in Unmanned Aerial System Imagery

In the computer vision space, deep neural networks (DNN) have gained tremendous popularity in recent years due to their ability to extract and classify visual features. As this technology has been applied to several different problem sets and applications, some of the shortcomings of the DNN have become apparent. Recently, researchers have increasingly found that DNN prefer to learn and extract texture from visual signals rather than shape and have observed that the more shape information is extracted, the better performance DNN can achieve. Meanwhile, from the digital image processing perspective, grayscale morphology works to extract shape information from visual signals using combinations of basic morphological operations. One such method is the Differential Morphological Profile (DMP), which performs morphological opening and closings with varied Structuring Element (SE) sizes and then takes the absolute difference between the resulting images. The DMP provides an opportunity to improve shape extraction in DNN, and with it, increase model robustness. To that end, a DMP-based neural network, or DMPNet, has been created to assist traditional DNN with extracting shape information by adding layers that perform DMP prior to the first convolutional layer. We use the DMPNet as a feature extractor to popular object detection algorithms, such as the Faster Region Proposal Convolutional Neural Network (Faster R-CNN), and apply it to a specialized application of object detection, i.e. maneuverability hazard detection in unmanned aerial system (UAS) imagery. The benefits of this approach include better explainability, lower training times, and produce models more tuned to shape information. We demonstrate that shape-based information helps models be more generalizable and applicable for object detection in changing contextual environments not seen during training.