Tracking focus of attention in meetings

This paper presents an overview of our work on tracking focus of attention in meeting situations. We have developed a system capable of estimating participants' focus of attention from multiple cues. In our system we employ an omni-directional camera to simultaneously track the faces of participants sitting around a meeting table and use neural networks to estimate their head poses. In addition, we use microphones to detect who is speaking. The system predicts participants' focus of attention from acoustic and visual information separately, and then combines the output of the audio- and video-based focus of attention predictors. In addition this work reports recent experimental results: In order to determine how well we can predict a subject's focus of attention solely on the basis of his or her head orientation, we have conducted an experiment in which we recorded head and eye orientations of participants in a meeting using special tracking equipment. Our results demonstrate that head orientation was a sufficient indicator of the subjects' focus target in 89% of the time. Furthermore we discuss how the neural networks used to estimate head orientation can be adapted to work in new locations and under new illumination conditions.