Deep Reinforcement Learning-Based Multirestricted Dynamic-Request Transportation Framework

Unmanned aerial vehicles (UAVs) are used in many areas where their usage is increasing constantly. Their popularity, therefore, maintains its importance in the technology world. Parallel to the development of technology, human standards, and surroundings should also improve equally. This study is developed based on the possibility of timely delivery of urgent medical requests in emergency situations. Using UAVs for delivering urgent medical requests will be very effective due to their flexible maneuverability and low costs. However, off-the-shelf UAVs suffer from limited payload capacity and battery constraints. In addition, urgent requests may be requested at an uncertain time, and delivering in a short time may be crucial. To address this issue, we proposed a novel framework that considers the limitations of the UAVs and dynamically requested packages. These previously unknown packages have source-destination pairs and delivery time intervals. Furthermore, we utilize deep reinforcement learning (DRL) algorithms, deep Q-network (DQN), proximal policy optimization (PPO), and advantage actor-critic (A2C) to overcome this unknown environment and requests. The comprehensive experimental results demonstrate that the PPO algorithm has a faster and more stable training performance than the other DRL algorithms in two different environmental setups. Also, we implemented an extension version of a Brute-force (BF) algorithm, assuming that all requests and environments are known in advance. The PPO algorithm performs very close to the success rate of the BF algorithm.