Improvising with Digital Auto-Scaffolding: How Mimi Changes and Enhances the Creative Process

Abstract

This chapter poses, and proposes some answers to, questions about the origins and nature of creativity when digital media takes an active role in the music-making process. The discussions are centered on François' Mimi (Multimodal Interaction for Musical Improvisation) system, which enables a musician to seed the computer with musical ideas and then improvise atop re-combinations of these ideas; the system provides the musician with visual foreknowledge of the machine's intent and review of the interaction. They extend to the different instantiations of, and extensions to, the Mimi system, which are designed with various interaction nuances in mind, and engender new forms of creativity. We review each Mimi version, from the original blue-and-white silhouette display, to the Scriabin-inspired varicolored panels, to the multi-paneled user-directed Mimi4x. In each scenario, we consider the impact of Mimi on the creative process and the resulting performance; specifically, we describe the interaction between a performer, the composer (when this is different from the performer), and the system, analyzing the techniques used to successfully negotiate a performance with Mimi, and the formal musical structures that result from this interaction 3 .

Playing with the Edge

“…Thus, a two-step bootstrapping method, allowing for the influence of a local contextual reset (spelling without influence of previous keys), was proposed to allow for abrupt key changes. The presence of sudden shifts in perceived tonality points to the need for a reset or memory wipe function in systems for tracking the dynamics of tonal perception, like the one implemented in the Mimi system, see Schankler, Chew, and François (2014). Performance timing also serves to clarify these transitions from one key to the next, as evidenced in Chew (2012) where Schnabel's tempo arcs are shown to consistently outline and reinforce the tonal groupings of the opening fifteen bars of Beethoven's Moonlight Sonata (Op.…”

COSMOS: Computational Shaping and Modeling of Musical Structures

“…However, machine-generated music is notoriously devoid of long-term structure or narrative interest. Some attempts have been made to address this problem through design of structural control into improvisation systems (Dubnov and Assayag, 2005 ; Francois et al, 2013 ; Schankler et al, 2014 ; Assayag, 2016 ; Nika et al, 2016 ). In Hyperscore (Farbood, 2001 ) a user-defined tension line controls the dissonance of the generated music.…”

A Functional Taxonomy of Music Generation Systems

“…A novelty of this system is that it allows the user to visually see the recent past and future generated music, so as to be aware of the musical context and to plan a response. The performer is also the operator of Mimi, controlling her learning rate, switching on and off the learning, and clearing her memory [Schankler et al 2014]. A variation on Mimi, Mimi4x [François et al 2013], allows the user to control four interacting Mimi instances and to structure an improvisation by deciding when and which of the four Mimi-generated streams start and stop, their re-combination rates, and the playback dynamic levels.…”