ARCHIMUSIC3D: Multimodal Playful Transformations between Music and Refined Urban Architectural Design

The commonly used 3D architectural design tools for urban environments fail to capture aspects of an urban design which are aesthetic as well as functional. This paper describes an innovative multimodal user interface through which an urban designer can work on the music transcription of a specific urban environment applying music compositional rules and filters in order to identify discordant entities, highlight imbalanced parts and make design corrections. The proposed platform offers sonification of an Urban Virtual Environment (UVE), simulating a real-world cityscape, offering visual interpretation and musically playful modification of its soundscape. The system presented offers: The ability to view and convert an urban street to music (ready to play) based on a specific grammar of converting architectural elements to musical elements; secondly, the ability to transform this music in order to 'harmonize' it based on musical rules as if composing and playing music; and finally, the prospect of converting back the aesthetically and harmonically "corrected" musical piece to a newly refined street or urban design, visualized in 3D. The presented platform comprises of three scenes, which compile the three main parts of the system's multimodal interface; e.g., the 3D scene, the Digital Audio Workstation (DAW) scene and the TouchOSC mobile controller. The purpose of this paper is to assist architects and urban designers in 1) identifying urban dissonances, 2) refining their design using musical rules and 3) interactively presenting the output both visually and acoustically.