Fast intra-coding unit partition decision in H.266/FVC based on deep learning

In the recent Future Video Coding (FVC) standard developed by the Joint Video Exploration Team (JVET), the quad-tree binary-tree (QTBT) block partition module makes use of rectangular block forms and additional square block sizes compared to quad-tree (QT) block partitioning module proposed in the predecessor High-Efficiency Video Coding (HEVC) standard. This block flexibility, induced with the QTBT module, significantly improves compression performance while it dramatically increases coding complexity due to the brute force search for Rate Distortion Optimization (RDO). To cope with this issue, it is necessary to consider the unique characteristics of QTBT in FVC. In this paper, we propose a fast QT partitioning algorithm based on a deep convolutional neural network (CNN) model to predict coding unit (CU) partition instead of RDO which enhances considerably QTBT performance for intra-mode coding. Based on a suitable diversified CU partition patterns database, the optimization process is set up with three levels CNN structure developed to learn the split or non-split decision from the established database. Experimental results reveal that the proposed algorithm can accelerate the QTBT block partition structure by reducing the intra-mode encoding time by an average of 35% with a bit rate increase of 1.7%, allowing its application in practical scenarios.