Low-Complexity Transform Design Using Hybrid Intra MTS

In recent video coding standards, the introduction of multiple transform selection (MTS) has significantly improved coding efficiency. The latest standard, Versatile Video Coding (VVC), adopted implicit MTS as an alternative tool to explicit MTS. This tool reduces the encoder complexity by inferring the optimal transform type from decoder-side information instead of performing an intensive rate-distortion search in the encoder. However, implicit MTS became less efficient in the enhanced compression model (ECM) due to the introduction of new primary transforms to explicit MTS, which drastically increased the number of possible horizontal and vertical transform pairs compared to VVC. This paper presents a new implicit MTS method that leverages a lookup table (LUT) to infer the best transform pair based on the block size and intra prediction mode. This LUT is used in a hybrid MTS scheme where explicit MTS is enabled only for blocks using matrix-based intra prediction (MIP). The proposed LUT-based hybrid method reduces the ECM complexity by 15% while retaining 60% and 83% of the MTS gain compared to the MTS method of ECM and VVC, respectively.