Playing Technique Recognition by Joint Time–Frequency Scattering

Wang, Changhong; Lostanlen, Vincent; Benetos, Emmanouil; Chew, Elaine

doi:10.1109/icassp40776.2020.9053474

Cited by 11 publications

(18 citation statements)

References 10 publications

(12 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This favorable result suggests that joint time-frequency scattering provides a useful feature map for learning similarities between instrumental playing techniques. In doing so, it is in line with a recent publication [74], in which the authors successfully trained a supervised classifier on joint time-frequency scattering features in order to detect and classify playing techniques from the Chinese bamboo flute (dizi). However, the originality of our work is that relies purely on auditory information (i.e., timbre similarity judgments), and does not require any supervision from the symbolic domain.…”

Section: Best Performing Systemsupporting

confidence: 53%

Time–frequency scattering accurately models auditory similarities between instrumental playing techniques

Lostanlen

El-Hajj

Rossignol³

et al. 2021

J AUDIO SPEECH MUSIC PROC.

Self Cite

View full text Add to dashboard Cite

Instrumentalplaying techniques such as vibratos, glissandos, and trills often denote musical expressivity, both in classical and folk contexts. However, most existing approaches to music similarity retrieval fail to describe timbre beyond the so-called “ordinary” technique, use instrument identity as a proxy for timbre quality, and do not allow for customization to the perceptual idiosyncrasies of a new subject. In this article, we ask 31 human participants to organize 78 isolated notes into a set of timbre clusters. Analyzing their responses suggests that timbre perception operates within a more flexible taxonomy than those provided by instruments or playing techniques alone. In addition, we propose a machine listening model to recover the cluster graph of auditory similarities across instruments, mutes, and techniques. Our model relies on joint time–frequency scattering features to extract spectrotemporal modulations as acoustic features. Furthermore, it minimizes triplet loss in the cluster graph by means of the large-margin nearest neighbor (LMNN) metric learning algorithm. Over a dataset of 9346 isolated notes, we report a state-of-the-art average precision at rank five (AP@5) of .%. An ablation study demonstrates that removing either the joint time–frequency scattering transform or the metric learning algorithm noticeably degrades performance.

show abstract

Section: Best Performing Systemsupporting

confidence: 53%

Time–frequency scattering accurately models auditory similarities between instrumental playing techniques

Lostanlen

El-Hajj

Rossignol³

et al. 2021

J AUDIO SPEECH MUSIC PROC.

Self Cite

View full text Add to dashboard Cite

show abstract

“…This favorable result suggests that joint time-frequency scattering provides a useful feature map for learning similarities between instrumental playing techniques. In doing so, it is in line with a recent publication [71], in which the authors successfully trained a supervised classifier on joint time-frequency scattering features in order to detect and classify playing techniques from the Chinese bamboo flute (dizi). However, the originality of our work is that Φ relies purely on auditory information (i.e., timbre similarity judgments), and does not require any supervision from the symbolic domain.…”

Section: Evaluation Metricsupporting

confidence: 53%

Time-Frequency Scattering Accurately Models Auditory Similarities Between Instrumental Playing Techniques

Lostanlen,

El-Hajj,

Rossignol

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Instrumental playing techniques such as vibratos, glissandos, and trills often denote musical expressivity, both in classical and folk contexts. However, most existing approaches to music similarity retrieval fail to describe timbre beyond the so-called "ordinary" technique, use instrument identity as a proxy for timbre quality, and do not allow for customization to the perceptual idiosyncrasies of a new subject. In this article, we ask 31 human subjects to organize 78 isolated notes into a set of timbre clusters. Analyzing their responses suggests that timbre perception operates within a more flexible taxonomy than those provided by instruments or playing techniques alone. In addition, we propose a machine listening model to recover the cluster graph of auditory similarities across instruments, mutes, and techniques. Our model relies on joint time-frequency scattering features to extract spectrotemporal modulations as acoustic features. Furthermore, it minimizes triplet loss in the cluster graph by means of the large-margin nearest neighbor (LMNN) metric learning algorithm. Over a dataset of 9346 isolated notes, we report a state-of-the-art precision at rank five (P@5) of 99.0% ± 1. An ablation study demonstrates that removing either the joint time-frequency scattering transform or the metric learning algorithm noticeably degrades performance.

show abstract

“…Music deconstruction includes research about melody, music spectral characteristics, the correlation of different types of music genres, and so on. Wang et al [17] did research about playing techniques recognization from the Dizi music spectrum. Yang et al [21] did a quantitative study of vibrato to compare erhu music and violin music.…”

Section: Related Workmentioning

confidence: 99%

Deconstruct and Reconstruct Dizi Music of the Northern School and the Southern School

Xie

2020

Preprint

View full text Add to dashboard Cite

Today's research on Chinese music technology is mainly focused on three aspects: data collection, music deconstruction, and music reconstruction. In this paper, a general method is proposed to collect Chinese music in the form of numbered musical notation, and a Dizi dataset is collected using this method. Based on the collected Dizi dataset, we conduct research on the Dizi music styles of the Northern school and the Southern School. Characteristics include melody and playing techniques of the two different music styles are deconstructed. A reconstruction example, music style transfer which includes melody transfer and playing techniques transfer is given and audience evaluation is done to evaluate the reconstruction results.

show abstract

Playing Technique Recognition by Joint Time–Frequency Scattering

Cited by 11 publications

References 10 publications

Time–frequency scattering accurately models auditory similarities between instrumental playing techniques

Time–frequency scattering accurately models auditory similarities between instrumental playing techniques

Time-Frequency Scattering Accurately Models Auditory Similarities Between Instrumental Playing Techniques

Deconstruct and Reconstruct Dizi Music of the Northern School and the Southern School

Contact Info

Product

Resources

About