Generalized Reinforcement Learning with Concept-Driven Abstract Actions

The standard reinforcement learning framework often faces challenges in a varying or evolving environment due to an inherent limitation in its representation. In particular, useful actions for decision making are often assumed to be a prefixed set prior to the learning process. Consequently, the derived policy in general lacks the ability to adapt to possible variations in the action outcomes or the action set itself without resorting to a substantial re-learning process. In addition, complexity in the state space modeling is often a bottleneck for standard learning methods. This paper proposes a new framework of reinforcement learning that enables the agent to formulate an action-oriented conceptual model while deriving the decision policy simultaneously. The new framework, Concept-Driven Learning Architecture (CDLA), formulates the abstract actions based on associating the correlated past decision history. Specifically, the kernel function, Gaussian process and spectral clustering mechanisms are combined into a functional clustering method to identify a set of coherent, concept-driven abstract actions using which the agent derives a control policy.