With the rapid development of technologies in networking and thin-client devices, such as remote desktop, video conferencing with documents or slides sharing, television programs and many curriculum videos playing in the Internet that contain mixtures of camera-captured content(CC) and screen content(SC), efficient screen content coding(SCC) becomes a hot topic in multimedia applications and has attracted increasing researcher attention from both academia and industry. Two international video coding standards include efficient SCC capability. One is High Efficiency Video Coding (HEVC), the other is the second-generation of the Audio Video Coding Standard(AVS2). In recent years, AVS is developing an AVS2 Screen and Mixed Content Coding(SMCC) extension(AVS2-SMCC). In our previous work, a 4:4:4 format universal string prediction (USP) algorithm integrated with the tradition 4:2:0 format block prediction and transform coding framework is used to code 4:4:4 format mixed content screen. To fully exploit both local and non-local, both general and special, and both complex and simple matching patterns with a variety of sizes and/or shapes and/or positions in a wide range of commonly seen screen content, a USP approach and its key technologies with three modes: general string(GS) mode, constrained string 1(CS1) mode, and constrained string 2(CS2) mode are proposed. The three constrained string modes are implemented with one of the three types of strings: offset string, coordinate string, and unpredictable pixel, or its combination of them. When using USP algorithm to code a coding unit, one of the three constrained modes which gets the minimum rate distortion value is selected to code the CU. Experimental results show that, for text and graphics with motion of AVS2-SMCC test sequences, USP algorithm achieves significantly improved coding efficiency in the All Intra configuration when compared with the latest HEVC-SCC extension. In order to further improve the coding efficiency in the Low Delay configuration and reduce the complexity, in this framework, according to the inherent inter-frame characteristics of the mixed content and the different characteristics for different inter-sub-coding modes, an optimized inter coding algorithm which consists of a 4:4:4 format inter-coding algorithm, a content adaptive weighted chroma distortion algorithm for different inter-sub-coding modes and an early terminate algorithm for inter-sub-coding modes pre-coding and coding tree unit partition, and a newly inter constrained string 1 mode (the search area of CS1 mode extends to the first frame in the reference picture queue) are proposed. Compared with the reference software RMD1.0 of AVS2-SMCC, for the category of text and graphics with motion in the AVS2-SMCC test sequences, the proposed algorithm achieves the average BD-rate reduction up to 14.35%, 90.73% and 88.85% respectively, in lossy low delay configuration, with a nearly 50.86% decrease in encoding runtime and 15.24% increase in decoding runtime. Especially for the video category of text and graphics with motion, the proposed algorithm achieves significantly improved coding efficiency not only in the All Intra configuration but also in the Low Delay configuration when compared with the HEVC-SCC extension. © 2019, Science Press. All right reserved.
