Improving the QoE of Real-Time Video Transmission: A Deep Lossy Transmission Paradigm

In conventional lossless transmission (CLT)-based real-time video transmission (RTVT), the user-perceived quality of the transmitted video frames decreases significantly even when there exists 1% packet loss. To improve the quality of experience of RTVT with lossy channels, we propose a semantic communication-based deep lossy transmission (DLT) paradigm by using a deep video semantic coding (DeepVSC) model to achieve end-to-end deep joint source-channel coding in RTVT, such that the quality of the recovered video frames can be significantly improved in lossy transmission scenarios by leveraging the strong data compression and error correction capabilities of DeepVSC. We present the basic framework of DLT, compare it with the CLT system, and an illustrative test shows that DLT can recover the image when packet loss rate (PLR) is 80%, while in CLT the images failed to be reconstructed when the PLR is 10%. We also summarize the research challenges of DLT to motivate more future research efforts in this area.