Energy-Efficient Mobile Crowdsensing by Unmanned Vehicles: A Sequential Deep Reinforcement Learning Approach

Mobile crowdsensing (MCS) is an attractive and innovative paradigm in which a crowd of users equipped with smart mobile devices (such as smartphones and iPads), and more recently unmanned vehicles (UVs, e.g., driverless cars and drones) conduct sensing tasks in mobile social networks by fully exploiting their carried diverse embedded sensors. These devices, especially UVs, are usually constrained by limited sensing range and energy reserve of devices, which contribute to the restriction of one single UV task performance, and thus UV collaborations are fully favored. In this article, we explicitly consider navigating a group of UVs to collect different kinds of data in a city, with the presence of multiple charging stations. Different from the existing approaches that solve the problem by forming a constrained optimization problem, we propose a novel sequential deep model called "PPO+LSTM," which contains a sequential model LSTM and is trained with proximal policy optimization (PPO), for assigning tasks and planning route. We evaluate our model in different network settings when comparing with other state-of-the-art solutions, and we also show the impact of important hyperparameters of our model. Results show that our solution outperforms all others in terms of energy efficiency, data collection ratio, and geographic fairness.