RAN Information-assisted TCP Congestion Control via DRL with Reward Redistribution

In this paper, we aim to propose a novel transmission control protocol (TCP) congestion control method from a cross-layer-based perspective and present a deep reinforcement learning (DRL)-driven method called DRL-3R (DRL for congestion control with Radio access network information and Reward Redistribution) so as to learn the TCP congestion control policy in a superior manner. In particular, we incorporate the RAN information to timely grasp the dynamics of RAN. Meanwhile, we relax the implicit assumption [i.e., the feedback of one specific data-sending rate controlling action returns at a round trip time (RTT) after the action is applied] in previous researches by redistributing the rewards and evaluating the merits of actions more accurately. Experiment results show that besides maintaining a reasonable fairness, DRL-3R significantly outperforms classical congestion control methods (e.g., TCP Reno, Westwood, Cubic, and BBR) on network utility by achieving a relatively high throughput while reducing delay in various network environments.