Non-negative dynamical system with application to speech and audio

Abstract

Non-negative data arise in a variety of important signal processing domains, such as power spectra of signals, pixels in images, and count data. This paper introduces a novel non-negative dynamical system (NDS) for sequences of such data, and describes its application to modeling speech and audio power spectra. The NDS model can be interpreted both as an adaptation of linear dynamical systems (LDS) to non-negative data, and as an extension of non-negative matrix factorization (NMF) to support Markovian dynamics. Learning and inference algorithms were derived and experiments on speech enhancement were conducted by training sparse non-negative dynamical systems on speech data and adapting a noise model to the unknown noise condition. Results show that the model can capture the dynamics of speech in a useful way. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. ABSTRACT Non-negative data arise in a variety of important signal processing domains, such as power spectra of signals, pixels in images, and count data. This paper introduces a novel non-negative dynamical system (NDS) for sequences of such data, and describes its application to modeling speech and audio power spectra. The NDS model can be interpreted both as an adaptation of linear dynamical systems (LDS) to non-negative data, and as an extension of non-negative matrix factorization (NMF) to support Markovian dynamics. Learning and inference algorithms were derived and experiments on speech enhancement were conducted by training sparse non-negative dynamical systems on speech data and adapting a noise model to the unknown noise condition. Results show that the model can capture the dynamics of speech in a useful way.Index Terms-non-negative dynamical system (NDS), linear dynamical system (LDS), multiplicative innovations, non-negative matrix factorization (NMF), source separation.

Supervised non-euclidean sparse NMF via bilevel optimization with applications to speech enhancement

“…At testing, the proposed method, like standard NMF approaches, treats different time-frames independently, ignoring the temporal dynamics of speech signals. Recent studies have proposed regularized variants of NMF or PLCA trying to overcome this limitation, including co-occurrence statistics of the basis functions [3], smoothness of the activation coefficients [17], and learned temporal dynamics [5,18,19]. In all these methods the model is expressed as the minimization of a cost with a data fitting term and some structure-promoting penalties.…”

Polyphonic Sound Event Tracking Using Linear Dynamical Systems

“…With respect to previous work on combining NMF with LDS: while the methods of [25] [26] are able to provide a component activation matrix that is able to evolve smoothly over time, in the present work we are primarily interested in using the LDS in a supervised scenario, to provide a mapping between the observed 'noisy' output of an event detection system and the latent 'true' sound event output, which is not possible using the aforementioned methods.…”

“…Recently, two NMF-based models were proposed for speech denoising and separation tasks, which incorporated temporal constraints similar to those of an LDS. In [25], an extension of NMF was proposed which supported Markovian dynamics: the observation model operates similarly to standard NMF, while the latent dynamics capture statistical dependencies between time frames similarly to LDS. In [26], a dynamic NMF model is proposed, where the observation model is similar to NMF/PLCA and follows a multinomial distribution, and the encoding matrix dynamics are formulated using an autoregressive model.…”

Non-negative source-filter dynamical system for speech enhancement

“…The general trade-off is that discrete-state approaches [4,5] can be more precise, especially in their temporal dynamics, whereas continuous approaches [6,7] can be more flexible with respect to gain and subspace variability.…”

“…Discrete state models, such as HMMs, represent dynamics using discrete state transitions over time [4,11]. Continuous state Gaussian dynamical models, such as linear dynamical systems (LDSs), have long been studied [12], and recently rich models of continuous dynamics have been extended to the NMF family using gammadistributed models [6,7] in models known as non-negative dynamical systems (NDSs). There have also been combinations with discrete dynamics and NMF observation models [13].…”