Mapless Navigation with Deep Reinforcement Learning based on The Convolutional Proximal Policy Optimization Network

In recent years, mapless navigation with a deep reinforcement learning approach in Autonomous Mobile Robot has shown a considerable benefit in improving robot behavior flexibility. Specifically, the Robot adapts to complex constraints and performs well in various environments without the need for a predetermined map and route plan. However, several previous studies show a lack of stability in the training of deep reinforcement learning networks for mapless navigation. Moreover, the exploration and exploitation in a specific work environment play an essential role in improving mapless navigation performance, which needs to be carefully considered. From the aforementioned issues, as well as inspired by the Proximal Policy Optimization algorithm has shown that it does not only evaluates the advantages and disadvantages of policy but also prevents the policy from changing too much after each time weight update. In this paper, we propose to build a Convolutional Proximal Policy Optimization network for the Mapless Navigation problem. Furthermore, the use of Boltzmann's policy to help balance exploration and exploitation also contributes to the Robot's ability to explore more deeply in complex environments, and the performance of the mapless navigation problem is also significantly improved.