PopMAG

Abstract

In pop music, accompaniments are usually played by multiple instruments (tracks) such as drum, bass, string and guitar, and can make a song more expressive and contagious by arranging together with its melody. Previous works usually generate multiple tracks separately and the music notes from different tracks not explicitly depend on each other, which hurts the harmony modeling. To improve harmony, in this paper 1 , we propose a novel MUlti-track MIDI representation (MuMIDI), which enables simultaneous multi-track generation in a single sequence and explicitly models the dependency of the notes from different tracks. While this greatly improves harmony, unfortunately, it enlarges the sequence length and brings the new challenge of long-term music modeling. We further introduce two new techniques to address this challenge: 1) We model multiple note attributes (e.g., pitch, duration, velocity) of a musical note in one step instead of multiple steps, which can shorten the length of a MuMIDI sequence. 2) We introduce extra long-context as memory to capture long-term dependency in music. We call our system for pop music accompaniment generation as PopMAG. We evaluate PopMAG on multiple datasets (LMD, FreeMidi and CPMD, a private dataset of Chinese pop songs) with both subjective and objective metrics. The results demonstrate the effectiveness of PopMAG for multi-track harmony modeling and long-term context modeling. Specifically, PopMAG wins 42%/38%/40% votes when comparing with ground truth musical pieces on LMD, FreeMidi and CPMD datasets respectively and largely outperforms other state-ofthe-art music accompaniment generation models and multi-track MIDI representations in terms of subjective and objective metrics. CCS CONCEPTS • Computing methodologies → Artificial intelligence; • Applied computing → Sound and music computing.

DisCover: Disentangled Music Representation Learning for Cover Song Identification

“…An effective musical representation is essential for learning different music-related tasks, such as music classification [7,27,36,48,55,56], cover song identification [56,58,62,63], music generation [17,18,28,40]. Most of them rely on large amounts of labeled datasets to learn music representations.…”

Byte Pair Encoding for Symbolic Music

“…• embedding pooling such as Compound Word (Hsiao et al, 2021) et al, 2021), PopMag (Ren et al, 2020), Sym-phonyNet (Liu et al, 2022) or MMT (Dong et al, 2023). Embeddings of several tokens are merged with a pooling operation.…”

“…the quality of generated music or the accuracy of classification tasks, and; 2) the efficiency of the models. The former is tackled with more expressive representations (Huang and Yang, 2020;Kermarec et al, 2022;von Rütte et al, 2023;Fradet et al, 2021), and the latter by representations based on either token combinations (Payne, 2019;Donahue et al, 2019), or embedding pooling (Hsiao et al, 2021;Zeng et al, 2021;Ren et al, 2020;Dong et al, 2023), which reduce the overall sequence length.…”

Attention-based speech feature transfer between speakers

“…However, with recent advancements in the field of speech synthesis, deep learning-based approaches have gained significant traction. This means that instead of relying on the conventional approach comprising multiple subprocesses, there has been a notable shift toward the development of end-to-end TTS technology (Ren et al, 2020), which is supported by trained models.…”