FIGARO: Generating Symbolic Music with Fine-Grained Artistic Control

Abstract

Generating music with deep neural networks has been an area of active research in recent years. While the quality of generated samples has been steadily increasing, most methods are only able to exert minimal control over the generated sequence, if any. We propose the self-supervised description-to-sequence task, which allows for fine-grained controllable generation on a global level. We do so by extracting high-level features about the target sequence and learning the conditional distribution of sequences given the corresponding high-level description in a sequence-tosequence modelling setup. We train FIGARO (FIne-grained music Generation via Attentionbased, RObust control) by applying descriptionto-sequence modelling to symbolic music. By combining learned high level features with domain knowledge, which acts as a strong inductive bias, the model achieves state-of-the-art results in controllable symbolic music generation and generalizes well beyond the training distribution.

Co-Performing Music with AI: Real-Time Performance Control Using Speech and Gestures

“…Deep learning is commonly used for music generation [8]. The most popular architectures are transformers for learning long-term sequential musical patterns [31,32,33] and variational autoencoders for unsupervised style encoding and control [34,35,36,37]. The models provide offline control over performance style [35,32] or performance parameters [36,37].…”

“…The most popular architectures are transformers for learning long-term sequential musical patterns [31,32,33] and variational autoencoders for unsupervised style encoding and control [34,35,36,37]. The models provide offline control over performance style [35,32] or performance parameters [36,37]. Recent works aim to generate music from descriptions [37] and text [38], as an intuitive way for humans to express themselves musically.…”

“…The models provide offline control over performance style [35,32] or performance parameters [36,37]. Recent works aim to generate music from descriptions [37] and text [38], as an intuitive way for humans to express themselves musically. Our work builds on these advances, focusing on real-time and interactive music performance generation that allows humans to participate directly in the creative process.…”

Using a Language Model to Generate Music in its Symbolic Domain while Controlling its Perceived Emotion

“…In a slightly different approach, FIGARO [38] also tackled the problem of controlled music generation in its symbolic domain by relying on transformer-based structures, but it introduces human interpretable and learned descriptions to guide the generative process. The former is determined by a hand-crafted algorithm, where features such as time signature, instruments, chords and note density are some of the quantities determining the descriptions.…”