Throughput Prediction-Enhanced RL for Low-Delay Video Application

Maximizing user quality of experience (QoE) is the ultimate goal of video players, and adaptive bitrate (ABR) is recognized as one of the most effective solutions. Approaches employing reinforcement learning (RL) have performed well as hybrid ABR algorithms, due to the ability to learn autonomously. However, throughput, which plays a crucial role in low-delay video streaming, is difficult to predict simply in mobile and wireless networks, and the inaccurately predicted throughput can lead to the wrong selection of bitrates. Worse, the general RL approaches are prone to frequent bitrate switching due to bandwidth fluctuation. These obstacles make the RL-based ABR approach unable to truly reflect the user QoE. We propose TP-RL, an application that makes ongoing decisions to maximize user QoE. To realize this, TP-RL adopts three ideas: (i) It takes the RL neural network as the main body of decision-making, which will inherit the advantages of RL and improve on this basis; (ii) Explore Mogrifier LSTM for throughput prediction, and replace the throughput part in the state space of the original RL neural network with a prediction module; (iii) The decided bitrate is further processed to achieve better smoothness when the bandwidth fluctuates. The performance of TP-RL is evaluated in different experimental environments, and experiments show that it can improve