Open-Unmix - A Reference Implementation for Music Source Separation

Abstract

Music source separation is the task of decomposing music into its constitutive components, e.g., yielding separated stems for the vocals, bass, and drums. Such a separation has many applications ranging from rearranging/repurposing the stems (remixing, repanning, upmixing) to full extraction (karaoke, sample creation, audio restoration). Music separation has a long history of scientific activity as it is known to be a very challenging problem. In recent years, deep learning-based systems-for the first time-yielded high-quality separations that also lead to increased commercial interest. However, until now, no open-source implementation that achieves state-of-the-art results is available. Open-Unmix closes this gap by providing a reference implementation based on deep neural networks. It serves two main purposes. Firstly, to accelerate academic research as Open-Unmix provides implementations for the most popular deep learning frameworks, giving researchers a flexible way to reproduce results. Secondly, we provide a pre-trained model for end users and even artists to try and use source separation. Furthermore, we designed Open-Unmix to be one core component in an open ecosystem on music separation, where we already provide open datasets, software utilities, and open evaluation to foster reproducible research as the basis of future development.

Addressing The Confounds Of Accompaniments In Singer Identification

“…The paper proposes a new SID model extending from CRNN and involving the use of melody information by leveraging CREPE [6]. Also, a data augmentation method called shuffleand-remix is adopted to avoid the confounds from the accompaniments by using source separation [12]. Our evaluation shows that both melody information and data augmentation improve the result, especially the latter.…”

“…In contrast, in our work both the SS model and the SID model employ deep learning. Specifically, we use open-unmix [12], an open-source three-layer bidirectional deep recurrent neural network for SS. Moreover, we build upon our SID model based on the implementation of a convolutional recurrent neural network made available by Nasrullah and Zhao [17], which attains the highest song-level F1-score of 0.67 on the per-album split of the artist20 dataset [18], a standard dataset for SID.…”

“…For simplicity, we use the same design for the mel-spectrogram branch and the melody contour branch. Second, instead of using the mel-spectrogram of the mixture audio recordings, we employ open-unmix [12] to remove the instrumental part of the music, and use the proposed data augmentation technique to increase the size of the training data, as described below.…”

“…Vocal-only: The vocal tracks separated by open-unmix [12]. In other words, all the accompaniments are removed.…”

Meta-Learning Extractors for Music Source Separation

“…A major design choice in music source separation models is whether to (1) train a separate model for each instrument [12], (2) to use a single class-conditional model, or (3) to use an instrument agnostic approach [16]. Our approach aims to combine the advantages of the first two; the high-precision of independent models, with improved optimization via parameter sharing in single models.…”