Human Intention Recognition using Markov Decision Processes

Human intention recognition in human-robot interaction (HRI) has been a papular topic. This paper presents a human-intention recognition framework using Markov decision processes (MDPs). The framework is composed of the object and motion layers. The object and motion layers obtain the object information and human hand gestures, respectively. The information extracted from the both layers is used to represent the state in the MDPs. To learn human intention to accomplish tasks, a frequency-based reward function in the MDPs is proposed. It assists the MDPs to converge to the policy that corresponds to the frequency of the task that has been performed. In our experiments, four tasks that were trained in different numbers of trial of pouring water and making coffee were used to validate the proposed framework. With the frequency-based reward function, the plausible intentional actions in certain states were distinguishable from the ones using the default reward function.