Novel Statistical Modeling, Analysis and Implementation of Rate-Distortion Estimation for H.264/AVC Coders

In H.264/advanced video coding, the encoder employs the rate-distortion optimization (RDO) to select the optimal coding mode of each block. Although it is effective to employ the RDO technique for mode decision, the computation load increases drastically. To reduce the computation complexity of the RDO technique, in this paper, we propose efficient algorithms for the estimation of block-level rate and distortion. For rate estimation, we model the transform coefficients with accurate generalized Gaussian distributions, and the weighted sum of absolute quantized transform coefficients is proposed as an efficient rate estimator, where the weights provide an implicit mechanism for evaluating different contributions of different frequency components to the coding bits. For distortion estimation, we first analyze the origins of distortion thoroughly. Then a direct relationship between the discarded bits in quantization and the distortion is explored. According to this investigation, a simple and efficient algorithm is proposed for the distortion estimation. With above proposed algorithms, the RDO technique can be efficiently implemented in a low-complexity way. Extensive experimental results demonstrate that, compared with the original RDO implementation, the proposed algorithms achieve about 32% reduced total encoding time with ignorable coding performance degradation.