Deep Learning-Based Outdoor Object Detection Using Visible and Near-Infrared Spectrum

Object detection is one of the essential branches of computer vision. However, detecting objects in the natural scene is challenging due to various reasons, for example, different sizes of objects, overlapping and similarities of colour, the texture of different objects, etc. The visible spectrum is not suited for standard computer vision tasks in many real-life scenarios. In low visibility settings, moving outside the visible spectrum range, such as to the thermal spectrum or near-infrared (NIR) imaging, is significantly more beneficial. For the object detection task in this study, we used photos from both the RGB and NIR spectrums. The purpose of this paper is to see if it's possible to use the information offered by the near-infrared (NIR) spectrum in conjunction with the visible band for object detection because they both have multimodal information. For example, because near-infrared wavelengths are less prone to haze and distortion, some visually indistinguishable things in the RGB spectrum can be spotted in the NIR image. We gathered a well-organized dataset of outdoor scenes in three spectra: visible (RGB), near-infrared (NIR), and thermal to train such a multispectral object recognition system. For the experiments, we use the YOLOv3 algorithm to train and evaluate our object detection models for NIR and RGB images separately, then train the model with four-channel input (3 channels from RGB images and one channel from NIR images) and the corresponding annotations to see if the model's performance improves even more in detecting the underlying objects. To determine the effectiveness of our approach, we conducted trials on YOLOv4 and SSD models and compared our results with existing related state-of-the-art models.