High performance resolution scalable video coding via all-phase motion compensated prediction of wavelet coefficients

How to develop a multi-resolution representation of video signals for both efficient and scalable coding? What are fundamental advantages and limitations of a resolution scalable video coder? How to make use of the scalability features of a video codec to compromise other requirements (e.g. delay)? These are the central issues we address in this paper. We first demonstrate the importance of resolving the phase uncertainty on the efficiency of motion-compensated prediction (MCP) in the wavelet domain. Improved understanding of the relationship between phase associated with any wavelet transform (WT) and motion accuracy of MCP motivates us to develop a novel multi-resolution representation for video signals. The salient feature of our new representation is that MCP can be performed both effectively and independently at different resolutions. We apply previous theoretical results on fractional-pel MCP to analyze the sacrifice of coding efficiency due to the resolution scalability constraint. We also investigate the issue of delay in video coding and propose a framework of trading delay with spatial resolution. A video decoder can display a low-resolution frame with low delay first and then gracefully enhance the frame resolution as the delay increases. The low-delay resolution-scalable MCP-WT coder built upon our new wavelet-based multi-resolution representation of video signals has achieved significant Rate-Distortion improvements over previously-reported scalable coders in the literature and even outperforms non-scalable MPEG-2 coder by 1-2dB at the bit rate of 2-9Mbps.