Generating Music Medleys via Playing Music Puzzle Games

Abstract

Generating music medleys is about finding an optimal permutation of a given set of music clips. Toward this goal, we propose a self-supervised learning task, called the music puzzle game, to train neural network models to learn the sequential patterns in music. In essence, such a game requires machines to correctly sort a few multisecond music fragments. In the training stage, we learn the model by sampling multiple non-overlapping fragment pairs from the same songs and seeking to predict whether a given pair is consecutive and is in the correct chronological order. For testing, we design a number of puzzle games with different difficulty levels, the most difficult one being music medley, which requiring sorting fragments from different songs. On the basis of state-of-the-art Siamese convolutional network, we propose an improved architecture that learns to embed frame-level similarity scores computed from the input fragment pairs to a common space, where fragment pairs in the correct order can be more easily identified. Our result shows that the resulting model, dubbed as the similarity embedding network (SEN), performs better than competing models across different games, including music jigsaw puzzle, music sequencing, and music medley. Example results can be found at our project website, https://remyhuang.github.io/DJnet.

Few-Shot Emergency Siren Detection

“…[27,28], acoustic event detection with meta-learning models [21,29,30] shows the effectiveness of these approaches in generalization to new audio events, outperforming supervised solutions based on fine-tuned convolutional neural networks. Five different few-shot learning methods [21,[31][32][33][34], improved with an attentional similarity module to detect transient events, are applied to sound event recognition in [35]. The effectiveness of few-shot techniques in sound event detection has led to the development of strategies to extend their application to increasingly challenging tasks, such as multi-label classification [36], rare sound event detection [37], continual learning [38], unsupervised and semi-supervised learning approaches [39], and sound localization [40].…”

Modeling the Compatibility of Stem Tracks to Generate Music Mashups

“…This may be different from conventional settings such as learning a representation of signals or their temporal coherence(Misra, Zitnick, and Hebert 2016;Huang, Chou, and Yang 2018).…”