Energy-Aware Deep Reinforcement Learning Scheduling for Sensors Correlated in Time and Space

Millions of battery-powered sensors deployed for monitoring purposes in a multitude of scenarios, e.g., agriculture, smart cities, industry, etc., require energy-efficient solutions to prolong their lifetime. When these sensors observe a phenomenon distributed in space and evolving in time, it is expected that collected observations will be correlated in time and space. This article proposes a deep reinforcement learning (DRL)-based scheduling mechanism capable of taking advantage of correlated information. The designed solution employs deep deterministic policy gradient (DDPG) algorithm. The proposed mechanism can determine the frequency with which sensors should transmit their updates, to ensure accurate collection of observations, while simultaneously considering the energy available. The solution is evaluated with multiple data sets containing environmental observations obtained in multiple real deployments. The real observations are leveraged to model the environment with which the mechanism interacts as realistically as possible. The proposed solution can significantly extend the sensors' lifetime and is compared to an idealized, all-knowing scheduler to demonstrate that its performance is near optimal. Additionally, the results highlight the unique feature of the proposed design, energy-awareness, by displaying the impact of sensors' energy levels on the frequency of updates.