Autorevise: Annotation refinement using motion signal patterns

Annotating a video for activity recognition, when precise, frame level activity localization is required, is time-consuming and difficult to accomplish with high accuracy. We propose a novel Autorevise approach for motion-based pattern recognition for improving the accuracy of activity labels of video data. This paper applies signal processing methods to motion features (e.g. speed of a subject as observed in the video) in order to identify shapes in the signals associated with activities to be classified. We evaluate this approach in the context of automatic and human-in-the-loop labeling of video data for C. elegans, an organism studied to understand functional neural activities. Our results show that the Autorevise method can identify the starting and ending frames of an activity, detect errors in human annotations, and improve the label consistency between pairs of annotators by up to 40%.