Hybrid model to detect zero quantized DCT coefficients in H.264

In H.264 coding, there are a large number of discrete cosine transform (DCT) coefficients of the prediction residue which are quantized to zeros. Therefore, it is desired to design a method which can early detect zero quantized DCT coefficients (ZQDCT) before implementing DCT and quantization (Q) and thus reduce redundant computations for H.264 coding. To achieve this, a hybrid model is proposed in this paper in order to predict ZQDCT coefficients. First, the Gaussian distribution is applied to study the integer DCT coefficients in H.264 and hence an adaptive scheme with multiple thresholds is derived to realize different types of DCT and Q implementations. Then the adaptive scheme is further optimized by considering a more efficient condition to sufficiently detect all-zero DCT blocks. As a result, a hybrid model is developed. Compared with other methods in the literature, the proposed hybrid model is able to detect more ZQDCT coefficients and hence reduce more computations for H.264 encoding. It is shown by experimental results that the proposed hybrid model can achieve the best performance in reducing computations and obtain almost the same rate-distortion (R-D) performance as the original encoder in the H.264 reference software JM9.5.