A Real-time Audio-to-audio Karaoke Generation System for Monaural Recordings Based on Singing Voice Suppression and Key Conversion Techniques

Tachibana, Hideki; Mizuno, Yuu; Sagayama, Shigeki

doi:10.2197/ipsjjip.24.470

Cited by 4 publications

(2 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A related goal is to design algorithms that can separate vocals and accompaniment, where all the instruments are considered as one source. Music source separation algorithms have been successfully used for predominant pitch tracking [1], accompaniment generation for Karaoke systems [2], or singer identification [3]. …”

Section: Introductionmentioning

confidence: 99%

Deep clustering and conventional networks for music separation: Stronger together

Luo

Zhuo

Hershey

et al. 2017

2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

134

View full text Add to dashboard Cite

Deep clustering is the first method to handle general audio separation scenarios with multiple sources of the same type and an arbitrary number of sources, performing impressively in speaker-independent speech separation tasks. However, little is known about its effectiveness in other challenging situations such as music source separation. Contrary to conventional networks that directly estimate the source signals, deep clustering generates an embedding for each time-frequency bin, and separates sources by clustering the bins in the embedding space. We show that deep clustering outperforms conventional networks on a singing voice separation task, in both matched and mismatched conditions, even though conventional networks have the advantage of end-to-end training for best signal approximation, presumably because its more flexible objective engenders better regularization. Since the strengths of deep clustering and conventional network architectures appear complementary, we explore combining them in a single hybrid network trained via an approach akin to multi-task learning. Remarkably, the combination significantly outperforms either of its components.

show abstract

Section: Introductionmentioning

confidence: 99%

Deep clustering and conventional networks for music separation: Stronger together

Luo

Zhuo

Hershey

et al. 2017

2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

134

View full text Add to dashboard Cite

show abstract

“…The two-stage HPSS system proposed by Tachibana et al in [224] proceeds the same way. It is an extension of the melody extraction approach in [225] and was applied for karaoke in [226]. It consists in using the optimization-based HPSS algorithm from [160], [215], [227], [228] at different frequency resolutions to separate the mixture into harmonic, percussive, and vocal components.…”

Section: E Cascaded and Iterated Methodsmentioning

confidence: 99%

An Overview of Lead and Accompaniment Separation in Music

Rafii

Liutkus

Stöter

et al. 2018

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

Popular music is often composed of an accompaniment and a lead component, the latter typically consisting of vocals. Filtering such mixtures to extract one or both components has many applications, such as automatic karaoke and remixing. This particular case of source separation yields very specific challenges and opportunities, including the particular complexity of musical structures, but also relevant prior knowledge coming from acoustics, musicology or sound engineering. Due to both its importance in applications and its challenging difficulty, lead and accompaniment separation has been a popular topic in signal processing for decades. In this article, we provide a comprehensive review of this research topic, organizing the different approaches according to whether they are model-based or data-centered. For model-based methods, we organize them according to whether they concentrate on the lead signal, the accompaniment, or both. For data-centered approaches, we discuss the particular difficulty of obtaining data for learning lead separation systems, and then review recent approaches, notably those based on deep learning. Finally, we discuss the delicate problem of evaluating the quality of music separation through adequate metrics and present the results of the largest evaluation, to-date, of lead and accompaniment separation systems. In conjunction with the above, a comprehensive list of references is provided, along with relevant pointers to available implementations and repositories.

show abstract

Mixing-Specific Data Augmentation Techniques for Improved Blind Violin/Piano Source Separation

Chiu

Hsiao²,

Yeh³

et al. 2020

2020 IEEE 22nd International Workshop on Multimedia Signal Processing (MMSP)

View full text Add to dashboard Cite

Blind music source separation has been a popular and active subject of research in both the music information retrieval and signal processing communities. To counter the lack of available multi-track data for supervised model training, a data augmentation method that creates artificial mixtures by combining tracks from different songs has been shown useful in recent works. Following this light, we examine further in this paper extended data augmentation methods that consider more sophisticated mixing settings employed in the modern music production routine, the relationship between the tracks to be combined, and factors of silence. As a case study, we consider the separation of violin and piano tracks in a violin piano ensemble, evaluating the performance in terms of common metrics, namely SDR, SIR, and SAR. In addition to examining the effectiveness of these new data augmentation methods, we also study the influence of the amount of training data. Our evaluation shows that the proposed mixing-specific data augmentation methods can help improve the performance of a deep learning-based model for source separation, especially in the case of small training data.

show abstract

A Real-time Audio-to-audio Karaoke Generation System for Monaural Recordings Based on Singing Voice Suppression and Key Conversion Techniques

Cited by 4 publications

References 30 publications

Deep clustering and conventional networks for music separation: Stronger together

Deep clustering and conventional networks for music separation: Stronger together

An Overview of Lead and Accompaniment Separation in Music

Mixing-Specific Data Augmentation Techniques for Improved Blind Violin/Piano Source Separation

Contact Info

Product

Resources

About