Towards Music-Aware Virtual Assistants

We propose a system for modifying spoken notifcations in a manner that is sensitive to the music a user is listening to. Spoken notifcations provide convenient access to rich information without the need for a screen. Virtual assistants see prevalent use in handsfree settings such as driving or exercising, activities where users also regularly enjoy listening to music. In such settings, virtual assistants will temporarily mute a user's music to improve intelligibility. However, users may perceive these interruptions as intrusive, negatively impacting their music-listening experience. To address this challenge, we propose the concept of music-aware virtual assistants, where speech notifcations are modifed to resemble a voice singing in harmony with the user's music. We contribute a system that processes user music and notifcation text to produce a blended mix, replacing original song lyrics with the notifcation content. In a user study comparing musical assistants to standard virtual assistants, participants expressed that musical assistants ft better with music, reduced intrusiveness, and provided a more delightful listening experience overall.