Musical and Conversational Artificial Intelligence

Abstract

Music production software often has complex interfaces and needs the user to know the basic musical know-how. In this paper, we present a conversational agent that allows creating music in a simplified way through voice-based interaction. Indeed, our agent can be configured and customized with simple and natural voice commands. In addition, it has some typically human cognitive skills to produce music: it listens to the user while singing a song and generates a melody by discovering and copying the patterns of her/his human voice. Technologically, the system is empowered by Google Dialogflow for conversation management and uses an advanced technique called abstract melody for music production. This Musical and Conversational Artificial Intelligence is an actual innovation since it does not require any preliminary knowledge about music and, consequently, includes professionals, but also children, beginners, and people with physical disease. CCS CONCEPTS• Human-centered computing → Natural language interfaces; • Applied computing → Sound and music computing.

DECI: A Tutorial on Designing Effective Conversational Interfaces

“…This tutorial is fundamentally relevant to the IUI community owing to its relation to intelligent conversational interfaces. More recently, researchers in IUI have been drawn to investigate CUIs for a variety of application domains, including learning a second language [25], sexual harassment prevention [5], qualitative interviews [3], and music production [4], among others. Furthermore, this tutorial will emphasize the distinctive characteristics of CUIs, ushering the IUI community to redefine and expand the scope of CUIs.…”

Boris: a Spoken Conversational Agent for Music Production for People with Motor Disabilities

“…Boris creates a melody that sounds correct from a harmonic point of view through an artificial emotional intelligence that exploits an advanced technique called abstract melody [4]. To do that, it combines the information obtained from both the conversation with the user and the analysis of her/his pitch while singing a song.…”

“…A conversational interface's strength is that it is accessible to all those who are verbal but with motor impairments. Besides, from literature, we know that speech impairments are widespread among people with some physical disabilities [17]; in response, Boris enables the interaction even by users with severe speech impairments by analyzing their vocalizations to obtain more than just the transcription: the system exploits a novel musical artificial intelligence that can imitate the typically human cognitive skills to produce music by using an advanced technique called abstract melody [4]. Consequently, the user can sing or hum a melody with her/his voice, and the system uses it to compose a song that sounds correct from a harmonic point of view.…”