Automatic Annotation for Human Activity and Device State Recognition Using Smartphone Notification

The process of human activity recognition needs to construct a model that has learned sensor data with annotations, i.e., groundtruth, label, or answer activity, in advance. Therefore, a large and diverse set of annotated data are needed to improve and evaluate model performance. Since it is difficult to judge the user's situation even after seeing the acceleration data, it is necessary to add an annotation to the collected acceleration data. In this paper we propose a method that estimates the user and device situations from the user's response to the notification generated by the device such as smartphone. User and device situations are estimated from the user's response time to the notification and the acceleration values in the device. Estimation result with high confidence is given to the sensor data as an annotation. Through the evaluation experiment, for seven kinds of annotation classes, an average precision of 0.769 and 0.963 for user-independent experiments and for user-dependent experiments were achieved, respectively.