A novel fast intra-mode decision algorithm for H.264/AVC

CABAC is the entropy coding method adopted in H.264/AVC main profile. RDO is the mode decision method commended by JVT, which can improve the performance of encoder markedly. However, the computation complexity of video encoding increases drastically. In this paper, we propose a novel fast intra-mode decision algorithm for H.264, based on transform-domain bit-rate estimation technique, using the estimated values of rate to calculate the Lagrange cost instead of the real values after entropy coding, thus avoiding the process of entropy coding in mode decision and reducing the computing time for video encoding. Besides, the intra-mode decisions for luma and chroma components are performed independently, which avoids the two-layer nested iteration in mode decision. Simulation results show that the proposed algorithm saves about 80% of total encoding time while the losses of PSNR and bit-rate are negligible for IIII sequence.