Recognizing Fine-Grained Home Contexts Using Multiple Cognitive APIs

To implement fine-grained context recognition affordable for general households, we are studying techniques that integrate image-based cognitive API and light-weight machine learning. Specifically, our method first captures images of a target space in different context, then sends them to the cognitive API. For each image, the API returns a set of words, called tags, representing concepts recognized in the picture. Regarding these tags as features of the target context, we apply the supervised machine learning. Our preliminary results with a commercial API showed that the overall accuracy was more than 90%, however, the accuracy decreased for contexts with multiple people (e.g., "General meeting", "Dining together" and "Play games"). The goal of this paper is to improve the recognition accuracy of such difficult contexts, with preserving the affordability to general households. In the proposed method, we use multiple cognitive APIs. For each API, we construct an independent recognition model. Then, the context is determined by majority voting among results of the independent models. Experimental evaluation with five commercial APIs shows that the ensemble of the five independent models achieved 98% of overall accuracy, where the individual models complement mutual limits of recognition.