Hierarchical Learning Approach for Age-of-Information Minimization in Wireless Sensor Networks

In this paper, we focus on a multi-user wireless network coordinated by a multi-antenna access point (AP). Each user can generate the sensing information randomly and report it to the AP. The freshness of information is measured by the age of information (AoI). We formulate the AoI minimization problem by jointly optimizing the users' scheduling and transmission control strategies. Moreover, we employ the intelligent reflecting surface (IRS) to enhance the channel conditions and thus reduce the transmission delay by controlling the AP's beamforming vector and the IRS's phase shifting matrices. The resulting AoI minimization becomes a mixed-integer program and difficult to solve due to uncertain information of the sensing data arrivals at individual users. By exploiting the problem structure, we devised a hierarchical deep reinforcement learning (DRL) framework to search for optimal solution in two iterative steps. Specifically, the users' scheduling strategy is firstly determined by the outer-loop DRL approach, and then the inner-loop optimization adapts either the uplink information transmission or downlink energy transfer to all users. Our numerical results verify that the proposed algorithm can outperform typical baselines in terms of the average AoI performance.