Pre-encoding based temporal dependent rate-distortion optimization for HEVC

Rate-distortion optimization (RDO) is a crucial technique in block-based hybrid video encoders, which determines the coding option for a certain coding unit to achieve optimal rate-distortion (R-D) performance. However, the current RDO method, which is applied in High Efficiency Video Coding (HEVC), is far from being globally optimal as it ignores the interaction among coding options. In fact, the current coding option not only determines the R-D point of the coded unit but also influences the achievable R-D performance of the subsequent coding units due to the spatial-temporal dependencies among the block/frame optimizations. In this paper, we first investigate the R-D characteristics, including operational R-D curves and R-D dependencies, in HEVC through several simulation experiments. It is shown that under the hierarchical fixed QP setting in the HEVC encoder, the temporal R-D dependence dominates the coding process, and the distortion of the reference frame has a strong impact on that of coded frames while less influence on rate. We then introduce distortion effect factors to quantitatively measure the temporal R-D dependency. The distortion effect factor at frame level reflects the correlation degree between the current coding frame and the previous coded frames, and the one at block level reflects the influence degree that the distortion of a block to that of subsequent blocks which refer to it in temporal domain. Finally, a pre-encoding based temporal dependent RDO is proposed, which adaptively adjusts quantization parameters and Lagrange multipliers according to the distortion effect factors. Experimental results show that under configurations of low-delay B and P frames, the proposed method can achieve 5.4% and 5.5% bit rate savings on average, respectively, with around 7% encoding complexity increment, compared with the original HEVC encoder.