Harmonic and Percussive Sound Separation and Its Application to MIR-Related Tasks

Ono, Nobutaka; Miyamoto, Ken-ichi; Kameoka, Hirokazu; Roux, Jonathan Le; Uchiyama, Yuuki; Tsunoo, Emiru; Nishimoto, Takuya; Sagayama, Shigeki

doi:10.1007/978-3-642-11674-2_10

Cited by 18 publications

(11 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The existing algorithm from Fuhrmann and Herrera [45] used typical hand-made timbral audio features with their framewise mean and variance statistics to train SVMs, and Bosch et al [44] improved this algorithm with source separation called FASST (Flexible Audio Source Separation Framework) [46] in a preprocessing step. In terms of precision, Fuhrmann and Herrera's algorithm showed the best performance for both the micro and the macro measure.…”

Section: A Comparison To Existing Algorithmsmentioning

confidence: 99%

Deep Convolutional Neural Networks for Predominant Instrument Recognition in Polyphonic Music

Han

Kim

Lee

2017

IEEE/ACM Trans. Audio Speech Lang. Process.

132

View full text Add to dashboard

Abstract-Identifying musical instruments in polyphonic music recordings is a challenging but important problem in the field of music information retrieval. It enables music search by instrument, helps recognize musical genres, or can make music transcription easier and more accurate. In this paper, we present a convolutional neural network framework for predominant instrument recognition in real-world polyphonic music. We train our network from fixed-length music excerpts with a single-labeled predominant instrument and estimate an arbitrary number of predominant instruments from an audio signal with a variable length. To obtain the audio-excerpt-wise result, we aggregate multiple outputs from sliding windows over the test audio. In doing so, we investigated two different aggregation methods: one takes the average for each instrument and the other takes the instrument-wise sum followed by normalization. In addition, we conducted extensive experiments on several important factors that affect the performance, including analysis window size, identification threshold, and activation functions for neural networks to find the optimal set of parameters. Using a dataset of 10k audio excerpts from 11 instruments for evaluation, we found that convolutional neural networks are more robust than conventional methods that exploit spectral features and source separation with support vector machines. Experimental results showed that the proposed convolutional network architecture obtained an F1 measure of 0.602 for micro and 0.503 for macro, respectively, achieving 19.6% and 16.4% in performance improvement compared with other state-of-the-art algorithms.

show abstract

Section: A Comparison To Existing Algorithmsmentioning

confidence: 99%

Deep Convolutional Neural Networks for Predominant Instrument Recognition in Polyphonic Music

Han

Kim

Lee

2017

IEEE/ACM Trans. Audio Speech Lang. Process.

132

View full text Add to dashboard

show abstract

“…Joint spatial and spectral modeling [5,6] and convolutive NMF have contributed to the reduction of the keyword error rate for small-vocabulary automatic speech recognition (ASR) from 44% down to as little as 8% in a strongly guided real-world domestic scenario involving knowledge of the speaker and his/her spatial position [32]. Finally, weakly guided separation of percussive and harmonic content in music has helped several music information retrieval (MIR) tasks, reducing for instance the relative error rate for chord recognition by 28% [33].…”

Section: Impact and Perspectivesmentioning

confidence: 99%

From Blind to Guided Audio Source Separation: How models and side information can improve the separation of sound

Vincent¹,

Bertin²,

Gribonval

et al. 2014

IEEE Signal Process. Mag.

126

103

View full text Add to dashboard

Audio is a domain where signal separation has long been considered as a fascinating objective, potentially offering a wide range of new possibilities and experiences in professional and personal contexts, by better taking advantage of audio material and finely analyzing complex acoustic scenes. It has thus always been a major area for research in signal separation and an exciting challenge for industrial applications. Starting with blind separation of toy mixtures in the mid 90's, research has progressed up to real-world scenarios today, with applications to speech enhancement and recognition, music editing, 3D sound rendering, and audio information retrieval, among others. This has mostly been made possible by the development of increasingly informed separation techniques incorporating knowledge about the sources and/or the mixtures at hand. For instance, speech source separation for remote conferencing can benefit from prior knowledge of the room geometry and/or the names of the speakers, while music remastering will exploit instrument characteristics and knowledge of sound engineers mixing habits. After a brief historical account, we provide an overview of recent and ongoing research in this field, illustrating a variety of models and techniques designed so as to guide the audio source separation process towards efficient and robust solutions.

show abstract

“…Before describing two-stage HPSS, let us first discuss HPSS [34], [35], [36], [50] briefly. HPSS is a technique that separates a spectrogram Y into two components below,…”

Section: Harmonic/percussive Sound Separation (Hpss)mentioning

confidence: 99%

A Real-time Audio-to-audio Karaoke Generation System for Monaural Recordings Based on Singing Voice Suppression and Key Conversion Techniques

Tachibana

Mizuno

Ono

et al. 2016

Journal of Information Processing

Self Cite

View full text Add to dashboard

This paper describes an automatic karaoke generation system, which can suppress the singing voice in audio music signals, and can also change the pitch of the song. Furthermore, this system accepts the streaming input, and it works in real-time. To the best of our knowledge, there have been no real-time audio-to-audio karaoke system that has the two functions above. This paper particularly describes the two technical components, as well as some comments on the implementation. In this system, the authors employed two signal processing techniques: singing voice suppression that is based on two-stage HPSS, a vocal enhancement technique that the authors proposed previously, and a pitch shift technique that is based on the spectrogram stretch and phase vocoder. The attached video file shows that the system works in real-time, and the sound quality may be practically acceptable.

show abstract

Harmonic and Percussive Sound Separation and Its Application to MIR-Related Tasks

Cited by 18 publications

References 9 publications

Deep Convolutional Neural Networks for Predominant Instrument Recognition in Polyphonic Music

Deep Convolutional Neural Networks for Predominant Instrument Recognition in Polyphonic Music

From Blind to Guided Audio Source Separation: How models and side information can improve the separation of sound

A Real-time Audio-to-audio Karaoke Generation System for Monaural Recordings Based on Singing Voice Suppression and Key Conversion Techniques

Contact Info

Product

Resources

About