Audio declipping via nonnegative matrix factorization

Abstract

Audio inpainting and audio declipping are important problems in audio signal processing, which are encountered in various practical applications. A number of approaches has been proposed in the literature to address these problems, most successful of which are based on sparsity of the audio signals in certain dictionary representations. Non-negative matrix factorization (NMF) is another powerful tool that has been successfully used in applications such as audio source separation. In this paper we propose a new algorithm that makes use of a low rank NMF model to perform audio inpainting and declipping. In addition to utilizing for the first time the NMF model to perform audio inpainting in presence of arbitrary losses in time domain, the proposed approach also introduces a novel way to enforce additional constraints on the signal magnitude in order to improve the performance in declipping applications. The proposed approach is shown to have a comparable performance with the state of the art dictionary based methods while providing a number of advantages.

Joint Audio Inpainting and Source Separation

“…3 It is assumed that 1 As opposed to [3], earlier works on audio inpainting with NMF/NTF models [9,11] cannot optimally address arbitrary losses in time domain, since the missing data are formulated in time frequency domain. 2 This work would be readily extended to the multi-channel case.…”

“…For sake of simplicity, we only consider the single-channel case here. 3 Throughout, letters with two primes, e.g., x ′′ , denote time domain signals, letters with one prime, e.g., x ′ denote framed and windowed-time domain signals and letters with no primes, e.g., x, denote complex-valued short-time Fourier transform (STFT) coefficients.…”

“…To this end, we extend our recent work on audio inpainting with application to declipping [3]. This approach, based on non-negative matrix factorization (NMF) performs as well or better than the state of the art group sparsity based methods such as [10].…”

“…This approach, based on non-negative matrix factorization (NMF) performs as well or better than the state of the art group sparsity based methods such as [10]. It builds on the recent successes of NMF [7] and non-negative tensor factorization (NTF) in audio inpainting [9,11,3] 1 . Since, NMF/NTF framework is also very powerful in source separation [13,5,8], it lends itself to addressing the joint problem of audio inpainting and audio source separation.…”

“…Extending [3], we estimate individual sources using a low rank NTF model, with the help of some temporal source activity information as in [8]. The proposed algorithm not only can perform audio inpainting and source separation sequentially (i.e., first inpaint the mixture, then separate the sources), but also can perform these two tasks jointly (i.e., simultaneously inpaint the mixture and separate the sources).…”