Automatic Action Segmentation and Continuous Recognition for Basic Indoor Actions Based on Kinect Pose Streams

One main difficulty in applying action recognition to practical applications is the need to segment beginnings and ends of actions in a continuous online monitoring process. This paper proposed a finite state machine (FSM) model for automatic action segmentation and recognition solution, based on pose streams in the form of skeleton joint data provided by Kinect. With the action recognition problem reframed as a state identification problem, the key solution to state identification hinges on detection of changing events, which signify the start of new action and the recognition of the underlying action. In that regard, a decision tree is constructed to detect these events based on the spatial positions and the changes of the skeleton data. In addition, to identify the current state or equivalently the action of a detected person, a fault observer is derived from the modeled action FSM. The fault observer does not only identify the initial state of action when the recognition process starts, but also serve the purpose of error recovery when the system loses track of ongoing events at times of intermittent sensor faults. Additionally, an AutoCorrect mechanism is presented to further enhance the accuracy of action recognition. To evaluate the proposed approach, an experiment with 300 participating subjects has been conducted for a total of 900 test sequences. The 98.63% correct identification result ensures the proposed approach a promising solution to constant action monitoring solution.