Context-Enhanced Meta-Reinforcement Learning with Data-Reused Adaptation for Urban Autonomous Driving

Autonomous driving (AD) has experienced rapid development in recent years, and the reinforcement learning (RL) pipeline in trial-and-error manner can surpass human driving ability. However, the poor performance in sample efficiency and generalization limits RL applying in the challenging urban traffic scenarios. In this paper, we build a context-enhanced meta-RL framework with data-reused adaptation for challenging urban AD. At both the meta-learning and adaptation stages, the context-enhanced state representation is designed to reduce the perceptual gap in variant urban scenarios, improving the sample efficiency and robustness. At adaptation stage, the metatraining data with context-enhanced features are reused through propensity estimation to constrain the optimization objective of new tasks, aiming to maintain the good driving performance of meta-trained policy and fast adapt to the new tasks. Extensive experiments are conducted in CARLA simulator with various urban environments and task settings. The learning curves and quantitative comparisons validate the good sample efficiency and generalization of our proposed method, with state-of-the-art driving performance on urban AD benchmarks.