Parallel online time warping for real-time audio-to-score alignment in multi-core systems

Alonso, Pedro; Cortina, Raquel; Rodriguez-Serrano, Francisco J.; Alonso-González, M.; Ranilla, José

doi:10.1007/s11227-016-1647-5

Cited by 14 publications

(27 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The software ReMAS (Real‐time Musical Accompaniment System), designed to track the reproduction of a musical piece with the aim to match the score position into its symbolic representation on a digital sheet, was presented in the work of Alonso et al ReMAS shows that it is possible to exploit efficiently several cores of an ARM® processor, or a GPU accelerator, reducing the processing time per frame in a few milliseconds in most of the cases. On the other hand, a parallel online DTW solution based on a client‐server architecture implemented for multicore architectures (86, 64, and ARM®) is proposed in another work of Alonso et al…”

Section: Proposed Frameworkmentioning

confidence: 99%

“…Both softwares have a high degree of similarity to the alignment stage. Thereby, the framework proposed for the alignment in this work aims to adapt, extend, and test the parallel heterogeneous algorithms of Alonso et al to the new problem.…”

Section: Proposed Frameworkmentioning

confidence: 99%

“…More information about the alignment described earlier can be found in the works of Carabias-Orti et al 17 and Rodriguez-Serrano et al 18 The software ReMAS (Real-time Musical Accompaniment System), designed to track the reproduction of a musical piece with the aim to match the score position into its symbolic representation on a digital sheet, was presented in the work of Alonso et al 24 ReMAS shows that it is possible to exploit efficiently several cores of an ARM ® processor, or a GPU accelerator, reducing the processing time per frame in a few milliseconds in most of the cases. On the other hand, a parallel online DTW solution based on a client-server architecture implemented for multicore architectures (86, 64, and ARM ® ) is proposed in another work of Alonso et al 25 Both softwares have a high degree of similarity to the alignment stage. Thereby, the framework proposed for the alignment in this work aims to adapt, extend, and test the parallel heterogeneous algorithms of Alonso et al 24,25 to the new problem.…”

Section: Alignment Stagementioning

confidence: 99%

“…On the other hand, a parallel online DTW solution based on a client-server architecture implemented for multicore architectures (86, 64, and ARM ® ) is proposed in another work of Alonso et al 25 Both softwares have a high degree of similarity to the alignment stage. Thereby, the framework proposed for the alignment in this work aims to adapt, extend, and test the parallel heterogeneous algorithms of Alonso et al 24,25 to the new problem.…”

Section: Alignment Stagementioning

confidence: 99%

See 3 more Smart Citations

Online score‐informed source separation in polyphonic mixtures using instrument spectral patterns

Muñoz-Montoro

Cortina

Combarro

et al. 2019

Comp and Math Methods

View full text Add to dashboard Cite

Soundprism is a real‐time algorithm to separate polyphonic music audio into source signals, given the musical score of the audio in advance. This paper presents a framework for a Soundprism implementation. A study of the sound quality of the online score‐informed source separation is shown, although a real‐time implementation is not carried out. The system is compound of two stages: (1) a score follower that matches a MIDI score position to each time frame of the musical performance; and (2) a source separator based on a nonnegative matrix factorization approach guided by the score. Real audio mixtures composed of an instrumental quartets were employed to obtain preliminary results of the proposed system.

show abstract

Section: Proposed Frameworkmentioning

confidence: 99%

Section: Proposed Frameworkmentioning

confidence: 99%

Section: Alignment Stagementioning

confidence: 99%

Section: Alignment Stagementioning

confidence: 99%

See 2 more Smart Citations

Online score‐informed source separation in polyphonic mixtures using instrument spectral patterns

Muñoz-Montoro

Cortina

Combarro

et al. 2019

Comp and Math Methods

View full text Add to dashboard Cite

show abstract

“…Regarding the SS, [22] uses a soft masking strategy based on a multipitch estimator, whereas the method in [6] used a frame-level NMF with a trained dictionary which is updated during the factorization. On the other hand, in our preliminary work [30], we proposed to use the source separation procedure presented in [13] along with the real-time implementation of the online alignment method from [31] presented in [32]. The signal model used in [30] was the same than in [13] but restricted to single-channel signals.…”

Section: Introductionmentioning

confidence: 99%

Online/offline score informed music signal decomposition: application to minus one

Muñoz-Montoro

Carabias-Orti

Canadas-Quesada

et al. 2019

J AUDIO SPEECH MUSIC PROC.

View full text Add to dashboard Cite

In this paper, we propose a score-informed source separation framework based on non-negative matrix factorization (NMF) and dynamic time warping (DTW) that suits for both offline and online systems. The proposed framework is composed of three stages: training, alignment, and separation. In the training stage, the score is encoded as a sequence of individual occurrences and unique combinations of notes denoted as score units. Then, we proposed a NMF-based signal model where the basis functions for each score unit are represented as a weighted combination of spectral patterns for each note and instrument in the score obtained from a trained a priori over-completed dictionary. In the alignment stage, the time-varying gains are estimated at frame level by computing the projection of each score unit basis function over the captured audio signal. Then, under the assumption that only a score unit is active at a time, we propose an online DTW scheme to synchronize the score information with the performance. Finally, in the separation stage, the obtained gains are refined using local low-rank NMF and the separated sources are obtained using a soft-filter strategy. The framework has been evaluated and compared with other state-of-the-art methods for single channel source separation of small ensembles and large orchestra ensembles obtaining reliable results in terms of SDR and SIR. Finally, our method has been evaluated in the specific task of acoustic minus one, and some demos are presented. which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.f ,t X(f , t)log X(f ,t) X(f ,t)− X(f , t) +X(f , t) β = 1 f ,t X(f ,t) X(f ,t)+ log X(f ,t) X(f ,t)

show abstract

An efficient musical accompaniment parallel system for mobile devices

2016

Self Cite

View full text Add to dashboard Cite

This work presents a software system designed to track the reproduction of a musical piece with the aim at match the score position into its symbolic representation on a digital sheet. Into this system, known as automated Musical Accompaniment System, the process of score alignment can be carried out real-time. A real-time score alignment, also known as score following, poses an important challenge due to the large amount of computation needed to process each digital frame and the very small time slot to process it. Moreover, the challenge is even greater since we are interested on handheld devices, i.e. devices characterized by both low power consumption and mobility. The results presented here show that it is possible to exploit efficiently several cores of an ARM R processor, or a GPU accelerator (presented in some SoCs from NVIDIA) reducing the processing time per frame under 10 ms in most of the cases.

show abstract

Parallel online time warping for real-time audio-to-score alignment in multi-core systems

Cited by 14 publications

References 9 publications

Online score‐informed source separation in polyphonic mixtures using instrument spectral patterns

Online score‐informed source separation in polyphonic mixtures using instrument spectral patterns

Online/offline score informed music signal decomposition: application to minus one

An efficient musical accompaniment parallel system for mobile devices

Contact Info

Product

Resources

About