Listen By Looking: a mobile application for augmented fruition of live music and interactive learning

This paper presents an ongoing project centered around the mobile application "Listen By Looking", realized with a graphic framework that allow developers to implement video games with specific tools for drawing music sheets, piano rolls and dialog windows with contextual information. The app is designed to entertain and teach music to users by making them listen to songs and then asking them to answer questions about the melodies and musical structures they have learned. The app has been assessed in laboratory by comparing the answers of people who have used the app with those of people who have listened the same music without the app. The results are encouraging for future developments of the app, with the possibility of using it during a public performance and with different games.