Parallel multichannel music source separation system

Muñoz-Montoro, Antonio J.; Suarez-Dou, D.; Carabias-Orti, Julio J.; Canadas-Quesada, F.; Ranilla, José

doi:10.1007/s11227-020-03282-2

Cited by 3 publications

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

The music demixing machine: toward real-time remixing of classical music

2023

View full text Add to dashboard Cite

Classical music, unlike popular music, is usually recorded live with close microphone techniques. For this reason, isolated tracks are not available to create the final mixture/stream, and so the mixing process requires greater effort. Source separation methods are a potential solution to this problem. However, current algorithms are not fast enough to yield real-time separation in professional setups with dozens of microphones and sources. In this paper, we propose a fast approach consisting of a panning-based multichannel non-negative matrix factorization model to separate classical music. We tested the system on real professional recordings, where we were able to reach real-time with very low latency and promising quality.

show abstract

The music demixing machine: toward real-time remixing of classical music

2023

View full text Add to dashboard Cite

show abstract

Efficient FPGA implementation for sound source separation using direction-informed multichannel non-negative matrix factorization

Diel,

Muñoz-Montoro,

Carabias-Orti

et al. 2024

J Supercomput

View full text Add to dashboard Cite

Sound source separation (SSS) is a fundamental problem in audio signal processing, aiming to recover individual audio sources from a given mixture. A promising approach is multichannel non-negative matrix factorization (MNMF), which employs a Gaussian probabilistic model encoding both magnitude correlations and phase differences between channels through spatial covariance matrices (SCM). In this work, we present a dedicated hardware architecture implemented on field programmable gate arrays (FPGAs) for efficient SSS using MNMF-based techniques. A novel decorrelation constraint is presented to facilitate the factorization of the SCM signal model, tailored to the challenges of multichannel source separation. The performance of this FPGA-based approach is comprehensively evaluated, taking advantage of the flexibility and computational capabilities of FPGAs to create an efficient real-time source separation framework. Our experimental results demonstrate consistent, high-quality results in terms of sound separation.

show abstract

An efficient parallel kernel based on Cholesky decomposition to accelerate Multichannel Non-Negative Matrix Factorization

Muñoz-Montoro

Carabias-Orti

Salvati

et al. 2022

Preprint

View full text Add to dashboard Cite

Multichannel Source Separation has been a popular topic, and recently proposed methods based on the local Gaussian model (LGM) have provided promising result despite its high computational cost when several sensors are used. The main reason being due to inversion of a spatial covariance matrix, with a complexity of \(O(I^3)\), being \(I\) the number of sensors. This drawback limits the practical application of this approach for tasks such as sound field reconstruction or virtual reality, among others. In this paper, we present a numerical approach to reduce the complexity of the Multichannel NMF to address the task of audio source separation for scenarios with a high number of sensors such as High Order Ambisonics (HOA) encoding. In particular, we propose a parallel multi-architecture driver to compute the multiplicative update rules in MNMF approaches. The proposed driver has been designed to work on both sequential and multi-core computers, as well as Graphics Processing Units (GPUs) and Intel Xeon coprocessors. The proposed software was written in C language and can be called from numerical computing environments. The proposed solution tries to reduce the computational cost of the multiplicative update rules by using the Cholesky decomposition and by solving several triangular equation systems.The proposal has been evaluated for different scenarios with promising results in terms of execution times for both CPU and GPU. To the best of our knowledge, our proposal is the first system that addresses the problem of reducing the computational cost of full-rank MNMF-based systems using parallel and high performance techniques.

show abstract

Parallel multichannel music source separation system

Cited by 3 publications

References 27 publications

The music demixing machine: toward real-time remixing of classical music

The music demixing machine: toward real-time remixing of classical music

Efficient FPGA implementation for sound source separation using direction-informed multichannel non-negative matrix factorization

An efficient parallel kernel based on Cholesky decomposition to accelerate Multichannel Non-Negative Matrix Factorization

Contact Info

Product

Resources

About