Audio-to-Score Alignment Using Deep Automatic Music Transcription

Simonetta, Federico; Ntalampiras, Stavros; Avanzini, Federico

doi:10.1109/mmsp53017.2021.9733531

Cited by 7 publications

(2 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Future works include further experiments to assess the proposed experimental strategy and to confirm the presented results, as well as the development of new technologies for feature fusion and context-aware music transcription, such as Paraconsistent Feature Engineering [24], Wavelet transforms, and novel audio-to-score alignment methods [53] Reproducibility and Supplementary Information…”

Section: Discussionmentioning

confidence: 80%

A Perceptual Measure for Evaluating the Resynthesis of Automatic Music Transcriptions

Simonetta,

Avanzini,

Ntalampiras

2022

Preprint

Self Cite

View full text Add to dashboard Cite

This study focuses on the perception of music performances when contextual factors, such as room acoustics and instrument, change. We propose to distinguish the concept of "performance" from the one of "interpretation", which expresses the "artistic intention". Towards assessing this distinction, we carried out an experimental evaluation where 91 subjects were invited to listen to various audio recordings created by resynthesizing MIDI data obtained through Automatic Music Transcription (AMT) systems and a sensorized acoustic piano. During the resynthesis, we simulated different contexts and asked listeners to evaluate how much the interpretation changes when the context changes. Results show that: (1) MIDI format alone is not able to completely grasp the artistic intention of a music performance; (2) usual objective evaluation measures based on MIDI data present low correlations with the average subjective evaluation. To bridge this gap, we propose a novel measure which is meaningfully correlated with the outcome of the tests. In addition, we investigate multimodal machine learning by providing a new score-informed AMT method and propose an approximation algorithm for the p-dispersion problem.

show abstract

Section: Discussionmentioning

confidence: 80%

A Perceptual Measure for Evaluating the Resynthesis of Automatic Music Transcriptions

Simonetta,

Avanzini,

Ntalampiras

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In these instances, researchers refer to modalities as multimodal information (16)(17)(18)(19) or simply "data types." This covers audio, video, images, and lyrics (20)(21)(22)(23)(24), as well as extracted features, body motion, physiological measurements (eye gaze, EEG, EDR, ECG, EMG, respiration), content, context (22), symbolic scores (25), MIDI (26), and even depth, thermal, and IMU data (27,28). Sometimes, it features "additional multimodal information" (16), like album covers, video clip links, and expert notes.…”

Section: Existing Definitionsmentioning

confidence: 99%

Multimodal Music Datasets? Challenges and Future Goals in Music Processing

Christodoulou,

Lartillot,

Jensenius

2024

Preprint

View full text Add to dashboard Cite

The term ``multimodal music dataset'' is often used to describe music-related data that represent music as a multimedia art form and multimodal experience. However, the term ``multimodality'' may mean different things in related disciplines, such as musicology, music psychology, and music technology. This paper proposes a definition of multimodality that works across different music disciplines. Many challenges are related to constructing, evaluating, and using multimodal music datasets. We provide a task-based categorization of these datasets and explore theoretical methodologies aimed at enhancing their future construction. Diverse data preprocessing methods are illuminated, highlighting their contributions to transparent music analysis. Additionally, evaluation metrics, methods, and benchmarks tailored for multimodal music processing tasks are scrutinized, empowering researchers to make informed decisions and facilitating cross-study comparisons.

show abstract

A perceptual measure for evaluating the resynthesis of automatic music transcriptions

Simonetta

Avanzini

Ntalampiras

2022

Multimed Tools Appl

Self Cite

View full text Add to dashboard Cite

This study focuses on the perception of music performances when contextual factors, such as room acoustics and instrument, change. We propose to distinguish the concept of “performance” from the one of “interpretation”, which expresses the “artistic intention”. Towards assessing this distinction, we carried out an experimental evaluation where 91 subjects were invited to listen to various audio recordings created by resynthesizing MIDI data obtained through Automatic Music Transcription (AMT) systems and a sensorized acoustic piano. During the resynthesis, we simulated different contexts and asked listeners to evaluate how much the interpretation changes when the context changes. Results show that: (1) MIDI format alone is not able to completely grasp the artistic intention of a music performance; (2) usual objective evaluation measures based on MIDI data present low correlations with the average subjective evaluation. To bridge this gap, we propose a novel measure which is meaningfully correlated with the outcome of the tests. In addition, we investigate multimodal machine learning by providing a new score-informed AMT method and propose an approximation algorithm for the p-dispersion problem.

show abstract

Audio-to-Score Alignment Using Deep Automatic Music Transcription

Cited by 7 publications

References 12 publications

A Perceptual Measure for Evaluating the Resynthesis of Automatic Music Transcriptions

A Perceptual Measure for Evaluating the Resynthesis of Automatic Music Transcriptions

Multimodal Music Datasets? Challenges and Future Goals in Music Processing

A perceptual measure for evaluating the resynthesis of automatic music transcriptions

Contact Info

Product

Resources

About