Soft Actor-Critic Algorithm for 360-Degree Video Streaming with Long-Term Viewport Prediction

In the tile-based 360-degree video streaming, it is essential to predict future viewport and to allocate higher bitrates to tiles inside the predicted viewport to optimize the Quality of Experience (QoE) of the users. However, the majority of existing work focuses on short-term viewport prediction, which is prone to rebuffering in dynamic network conditions. On the other hand, the recently developed on-policy Deep Reinforcement Learning (DRL)-based bitrate allocation approaches suffer from poor sample efficiency. To address these issues, in this paper we present a tile-based adaptive 360-degree video streaming system, named LS360, which consists of long-term viewport prediction and adaptive bitrate allocation. First, we propose a Long Short-Term Memory (LSTM)-based viewport prediction model to make use of the heatmap feature from all users' previous movement information and the target user's fixation movement feature to improve prediction accuracy. Next, we employ the off-policy Soft Actor-Critic (SAC) algorithm to make optimal tile bitrate allocation decisions by taking the predicted long-term viewport, playback buffer, and bandwidth-related information into account. Experiments on real-world datasets demonstrate that LS360 outperforms state-of-the-art streaming algorithms in terms of long-term viewport prediction accuracy and QoE under different bandwidth conditions.