Reinforcement Learning in Few-Shot Scenarios: A Survey

Reinforcement learning has a demand for massive data in complex problems, which makes it infeasible to be applied to real cases where sampling is difficult. The key to coping with these few-shot problems is knowledge generalization, and related algorithms are often called few-shot reinforcement learning (FS-RL). However, there lacks a formal definition and comprehensive analyses of few-shot scenarios and FS-RL algorithms. Therefore, after giving a uniform definition, we categorize few-shot scenarios into two types. The first type pursues more professional performance, while the other one pursues more general performance. In the process of knowledge transfer, few-shot scenarios usually have an obvious tendency to some type of knowledge. Based on this, we divide FS-RL algorithms into two types: the direct transfer case and the indirect transfer case. Thereafter, existing algorithms are discussed under this classification. Finally, we discuss future directions of FS-RL from the aspect of both theory and application.