Recovering a sequence of clear frames from a single motion-blurred image using correlation image sensor and temporal progressive learning strategy

Motion-blurred images are the result of an integration process, where instant light intensity is accumulated over the exposure time. Unfortunately, reversing this process is nontrivial. Firstly, integration destroys the temporal ordering of motion, resulting in ambiguity in the motion direction within a single motion-blurred image. Secondly, unlike conventional single-image deblurring, restoring a sequence of frames can divert the neural network's attention to each individual frame, which results in a decrease in the overall restoration quality of the entire sequence. To address the first problem, we leverage a crucial clue: the correlation image, generated by the three-phase correlation image sensor (3PCIS). This image effectively expresses motion information over the exposure time, which is essential for determining the motion direction of moving objects. We design a two-stream network to restore a sequence of sharp frames from a pair of motion-blurred images and their corresponding correlation images. For the second problem, we propose a temporal progressive learning (TPL) strategy that mitigates the performance degradation caused by network distraction by gradually increasing the number of restored clear frames during training. Experimental results demonstrate that our proposed method surpasses previous state-of-the-art methods on the GoPro public dataset and in real scenes.