Vehicle tracking in UAV video using multi-spectral spatiogram models

In this paper we present the results of applying a general purpose feature combination framework for tracking to the specific task of tracking vehicles in UAV data sets. In the fusion framework used (previously presented elsewhere(1)) vehicles' pixel-based features from multiple channels, specifically RGB and thermal IR, are split across separate individual spatiogram trackers. The use of spatiograms allows embedding of some spatial information into the models whilst also avoiding the exponential increase in computational load and memory requirements associated with the more commonly used histogram. This tracking framework is embedded in a complete system for detecting and tracking vehicles. The system first carries out pre-processing to ensure spatially and temporally aligned visible spectrum and IR data prior to tracking. Vehicle detection in the initial two frames is achieved by first compensating for camera motion, followed by frame differencing and post-processing (thresholding and size filtering) to identify vehicle regions. Each vehicle is then described by a bounding box and this is used to generate a set of spatiograms for each of the available data channels. The detected vehicle is then tracked using the spatiogram tracker framework. Results of experiments on a variety of UAV data sets indicate the promising performance of the overall system, even in the presence of significant illumination variation, partial and full occlusions and significant camera motion and focus change. Results are particularly encouraging given that we do not periodically re-initialise the detection phase and this points to the robustness of the tracking framework.