Deep representation learning for license plate recognition in low quality video images

License plate recognition is an important technology in many application scenarios such as traffic monitoring and vehicle management. Due to variations in viewpoint, illumination, motion-blur, and degradation during the imaging process, it is still a challenging problem to detect and recognize license plates in low quality video images. In this paper, we focus on efficient deep representation learning for license plate recognition, detection and tracking. For license plate recognition, we mainly investigate the configuration of different network structures. We design a novel backbone network structure called SACNN, which combines convolutional neural network (CNN) and self-attention mechanism to learn non-linear representations for the structural patterns of characters in low quality video images. The proposed license plate recognition model employs the SACNN backbone network, a Long Short-Term Memory (LSTM) encoder and a Transformer decoder. For license plate detection, a Transformer encoder-decoder based method is adopted. To tackle the variations in license plate appearances and perspectives, an image rectification method is incorporated by using a spatial transformer network. For license plate tracking, a multi-object tracking method is incorporated by using Kalman filtering and temporal matching to associate detected license plates in video frames. Experiments are mainly carried out on the public large-scale video-based license plate dataset (LSV-LP) to validate the proposed methods.