MetaIP: Meta-Network-Based Intra Prediction With Customized Parameters for Video Coding

Intra prediction is a vital tool in video coding that eliminates the spatial redundancy within a frame to enhance compression efficiency. Conventional intra prediction methods employ multiple directional prediction modes to describe textures in local areas. Recently, research on neural network-based intra prediction has achieved great success. The block-context pairs are divided into multiple clusters according to a predefined relationship, and a corresponding network is trained and applied for each cluster. However, the networks in these methods adopt fixed parameters to predict diverse image blocks, making it hard to cope with various textures in natural images. Inspired by recent works on parameter prediction, in this paper, we propose a meta-network-based intra prediction method, called MetaIP, that dynamically customizes the network parameters for each block sample in a given cluster. MetaIP consists of a meta-subnetwork and a prediction subnetwork. For an image block, the meta-subnetwork takes its neighboring reference pixels and some auxiliary information (e.g., quantization parameter) as inputs to generate customized parameters first. Then, the prediction subnetwork uses the customized parameters to infer the predicted block. MetaIP can generate multiple sets of network parameters corresponding to multiple modes for an image block. The optimal mode is determined by the rate-distortion optimization. MetaIP is integrated into VVC to assist or replace the directional prediction modes to evaluate its performance. The experimental results demonstrate that MetaIP with four prediction modes achieves an average of 3.84% and 1.96% bitrate saving for the luma component over VTM-17.0 when assisting or replacing VVC intra modes, respectively.