Learning to Navigate in 3D Virtual Environment Using Q-Learning

We implement a self-learning agent capable of navigating through an unknown 3-dimensional (3D) virtual environment. Leverage on Q-learning, an agent gradually learns to navigate the environment through self-exploratory. In contrast to previous works in this area, our state-value function is independent to an agent's position in the environment. Instead, the state-value is designed based on agent-visual perception (i.e. how the environment is perceived through 'raycasting'. This allows a greater flexibility for the agent to reuse its policy in different environment. This approach is also flexible since the Q-table does not grow according to the size of the environment. We investigate the flexibility of knowledge acquisition by examining its performance on different environment setup. The experiment results show that the agent successfully learns to navigate the environment.