Don't overlook any detail: Data-efficient reinforcement learning with visual attention

With the widespread application of visual reinforcement learning across various domains, the introduction of visual attention mechanisms aims to emulate human visual tasks, enabling deep models to focus on the crucial parts of images and enhancing model performance. However, in situations with limited data samples, solely introducing visual attention mechanisms can exacerbate overfitting in deep reinforcement learning (DRL), deteriorating performance. Herein, we propose a method called 'Don't overlook any detail (DOAD)' to tackle this issue. A two-step training strategy is proposed to increase the training frequency of the visual attention module while avoiding the specification of explicit tasks and fully acknowledging the pivotal role of visual attention in the learning process, rendering the model more adaptable to environmental changes. Furthermore, a conditional network reset method is proposed to simulate the flexibility observed inhuman learning processes, encouraging the model to adapt more flexibly to new information through regular reset mechanisms without excessively adhering to early knowledge. Finally, extensive experiments were conducted on 26 game environments within the Atari 100K environment. Compared to the baseline with the introduction of visual attention on the interquartile mean (IQM) from 0.44 to 0.37, the introduction of DOAD visual attention methods can improve the IQM to 0.70. DOAD elucidates the internal mechanisms of DRL and offers novel insights for applying visual attention mechanisms in DRL models under limited sample data contexts.