Detecting social interactions of the elderly in a nursing home environment

Social interaction plays an important role in our daily lives. It is one of the most important indicators of physical or mental changes in aging patients. In this article, we investigate the problem of detecting social interaction patterns of patients in a skilled nursing facility using audio/visual records. Our studies consist of both a "Wizard of Oz" style study and an experimental study of various sensors and detection models for detecting and summarizing social interactions among aging patients and caregivers. We first simulate plausible sensors using human labeling on top of audio and visual data collected from a skilled nursing facility. The most useful sensors and robust detection models are determined using the simulated sensors. We then present the implementation of some real sensors based on video and audio analysis techniques and evaluate the performance of these implementations in detecting interactions. We conclude the article with discussions and future work.