Multi-label Emotion Classification in Music Videos Using Ensembles of Audio and Video Features

Video as well as music are potent means to convey emotions. However, despite their importance in several applications, few works deal with the issue of emotion classification in videos. The main reason is possibly the lack of available databases. In this work we extend the CAL500 database by including music videos, since the CAL500 was originally proposed as an audio-only database. The main rationale here is that the music videos must be official as they were developed to convey the same emotion as the song. After adapting the database, we have extracted audio and video features to perform our computational experiments. Our main result is that there is a complementarity between the audio and video features as the best result was achieved using their combination.