Designing Deep Reinforcement Learning for Human Parameter Exploration

Abstract

Software tools for generating digital sound often present users with high-dimensional, parametric interfaces, that may not facilitate exploration of diverse sound designs. In this article, we propose to investigate artificial agents using deep reinforcement learning to explore parameter spaces in partnership with users for sound design. We describe a series of user-centred studies to probe the creative benefits of these agents and adapting their design to exploration. Preliminary studies observing users’ exploration strategies with parametric interfaces and testing different agent exploration behaviours led to the design of a fully-functioning prototype, called Co-Explorer, that we evaluated in a workshop with professional sound designers. We found that the Co-Explorer enables a novel creative workflow centred on human–machine partnership, which has been positively received by practitioners. We also highlight varied user exploration behaviours throughout partnering with our system. Finally, we frame design guidelines for enabling such co-exploration workflow in creative digital applications.

Conceptualising sound-driven design: an exploratory discourse analysis

“…A lexicon of 35 verbal descriptors of the salient morphological characteristics of sound (i.e., basic psychoacoustic features, timbre, and temporal descriptions) was validated and conceptualised in a pack of cards as well in a software interface to facilitate the communication between sound designers and non-experts [5]. Co-Explorer is a software tool that exploits reinforcement learning algorithms to enable creative human(s)machine partnerships in the exploration of high-dimensional, parametric sound spaces [34].…”

Artificial Intelligence in Music and Performance: A Subjective Art-Research Inquiry

“…However, most of these attempts have highlighted the inherent challenges of this approach due to the difficulty of grasping what AI can and cannot do [56]. In my work (first author), I am exploring how the computational learning mechanisms themselves can become interactive in order to foster exploration and human learning, as we recently explored in the specific domain of sound design [50].…”

Prototyping Machine Learning Through Diffractive Art Practice

“…In this context, we propose to look at the emergent field of human-centred ML, which applies user-centred and participatory design methods to design interactive machine learning tools [31,76,81], highlighting the qualitative concepts used by non-ML experts users to evaluate ML compared to quantitative concepts used by ML engineers [2]. Interestingly, Yang et al observed how small data approaches of interactive machine learning helped designers who are not ML experts to craft and experiment with ML to build functional systems [92].…”

“…Fiebrink et al's Wekinator [31,33] was designed to allow performers and musicians to craft sound interactions through demonstrations and allowed for incrementally act upon ML components as materials constituting the interaction design. Recently, Scurto et al 's Co-Explorer used interactive reinforcement learning to let people perform sound space exploration by communicating positive or negative reward data to an algorithmic agent [76]. Reinforcement learning was here used to craft sounds as well as exploration behaviours produced by the algorithmic agent.…”

“…We believe that art practices that use code and ML as creative material can be one such embodied approach to implement concrete prototypes of ML. In code-based art practice [55], artists approach ML not only as a technique to engineer logical concepts or functionalities-such as automatic classification or reward maximisation by learning agents-; they also approached ML as a computational material, whose raw properties, such as adaptive learning [20], model extrapolation [32], algorithmic exploration [76], or probabilistic uncertainty [9], can be crafted and experienced within installations, performances, and other hybrid embodiments [1,5,25,39,64]. By focusing on ML materiality, code-based art practice may thus help design research explore ML for what it is-i.e., a set of computational material possessing specific properties-rather than on what it is currently used for-e.g., a set of computational techniques contributing to socio-cultural discourses on artificial intelligence.…”