Towards Secrecy Energy-Efficient RIS aided UAV Network: A Lyapunov-Guided Reinforcement Learning Approach

Unmanned aerial vehicles (UAVs) are integrated into existing networks to enhance coverage, increase network capacity and provide ubiquitous access service. However, the channel in the UAV network is prone to noise and interference due to the complex environments. Reconfigurable intelligent surface (RIS), as an emerging technology in recent years, can be applied to the UAV network to establish the transmission environment by intelligibly adjusting signal characteristics, which can achieve significant gains in coverage and spectral efficiency. Thus, we consider RIS aided UAV networks for virtual reality (VR) content transmission under the presence of eavesdroppers, and maximize the time average sum secrecy energy efficiency (SEE) via adjusting UAV trajectory, beamforming matrix of UAV and RIS jointly by the deep reinforcement learning (DRL) approach. To eliminate the time correlation and the coupling of variables, we propose a Lyapunov guided decay twin-delayed deep deterministic policy gradient (TD3) scheme to tackle the decoupled problem. Simulations demonstrate the effectiveness of the proposed scheme and its outperformance in SEE compared with other benchmarks.