IR-MPE: A Long-Term Optical Flow-Based Motion Pattern Extractor for Infrared Small Dim Targets

In complex scenarios, the utilization of temporal motion information can improve the detection performance of infrared small and dim targets. However, existing multiframe methods only consider short-term motion information at each moment, which is difficult to capture reliable motion information for small and dim targets. In addition, existing multiframe data-driven methods generally utilize complex network structures which have longer inference times compared with the single-frame models of infrared target detection. Such a problem limits the applicability of the existing multiframe methods. In this article, we propose a long-term optical flow (OF)-based infrared small target motion pattern extractor (IR-MPE) to generate long-term OF energy (OFE) maps, which reflect the motion patterns of targets at the current moment. First, we design a long-term OF adaptive accumulation module (OFAAM) to adaptively control the update of current motion information and the retention of previous motion information. Second, we design an offset correction module (OCM), which is embedded in the OFAAM module to rectify the OFE from the previous frame. Meanwhile, the OCM also corrects the output of the previous frame to assist in the detection of the current frame. Embedding our IR-MPE module into the existing single-frame methods can easily extend them as multiframe methods. The only modification is adding an extra input channel of the first layer whose input is set as the concatenation of our OFE and the original infrared image. Such a simple structure can significantly improve the detection accuracy while maintaining fast inference speed. Extensive experiments on various public datasets show that our approach outperforms the state-of-the-art methods in challenging scenarios.