A Deep Look into Group Happiness Prediction from Images

We propose a Deep Neural Network (DNN) architecture to predict happiness displayed by a group of people in images. Our approach exploits both image context and individual facial information extracted from an image. The latter is explicitly modeled by Long Short Term Memory networks (LSTMs) encoding face happiness intensity and the spatial distribution of faces forming a group. LSTM outputs are combined with image context descriptors, to obtain the final group happiness score. We thoroughly evaluate our approach on the recently proposed HAPPEI dataset for group happiness prediction. Our results show that the proposed architecture outperforms a baseline CNN trained to predict group happiness directly from an image. Our method also shows an improvement of approximately 30% over the HAPPEI dataset's baseline on the validation set.