SSIM-Based Game Theory Approach for Rate-Distortion Optimized Intra Frame CTU-Level Bit Allocation

A structural similarity (SSIM)-based game theory (GT) approach is proposed for rate-distortion (R-D) optimized CTU-level bit allocation in high efficiency video coding (HEVC). First, a SSIM-based bargaining game is formulated and the Nash bargaining solution (NBS) is proposed, in which a SSIM-based initial minimum utility is defined. Second, we propose a two-stage remaining bit refinement-based bit allocation scheme. The optimization scheme of the SSIM-based bargaining game sufficiently considers the different R-D characteristics of coding tree units (CTUs), in which the feasible utility set is proved to be convex based on the proposed SSIM-based utility and R-SSIM model. Compared with the other state-of-the-art CTU-level bit allocation methods, the R-D performance improvements on Bjontegaard delta bit-rate (BD-BR), Bjontegaard delta peak-signal-to-noise-ratio (BD-PSNR), and BD-SSIM metrics of the proposed method can averagely achieve significant gains, respectively. The achieved R-D performance gains have been very close to the coding performance limits from the FixedQP method. Moreover, the proposed SSIM-GT method also maintains good performances on quality smoothness, bit rate accuracy, and encoding complexity.