Prioritized Environment Configuration for Drone Control with Deep Reinforcement Learning

In reinforcement learning, first, the agent collects experiences by interacting with the environment through trial-and-errors (experience collection stage) and then learns from the collected experiences (learning stage). This two-stage training process repeats until the agent solves a given task and requires a lot of experience, computation power, and time for training the agent. Therefore, many studies are conducted to improve the training speed and performance to mitigate them, focusing on the learning stage. This paper focuses on the experience collection stage and proposes a prioritized environment configuration that prioritizes and stochastically samples the effective configuration for initializing the environment for every episode. Therefore, we can provide the environments initialized with the configuration suitable for effective experience collection to the agent. The proposed algorithm can complement the reinforcement learning algorithms that focus on the learning stage. We have shown speed and performance improvement by applying the prioritized environment configuration to an autonomous drone flight simulator. In addition, the results show that the proposed algorithm works well with both on-policy and off-policy reinforcement learning algorithms in distributed framework with multiple workers.