TaSNet: Time-Domain Audio Separation Network for Real-Time, Single-Channel Speech Separation

Abstract

Robust speech processing in multi-talker environments requires effective speech separation. Recent deep learning systems have made significant progress toward solving this problem, yet it remains challenging particularly in real-time, short latency applications. Most methods attempt to construct a mask for each source in time-frequency representation of the mixture signal which is not necessarily an optimal representation for speech separation. In addition, time-frequency decomposition results in inherent problems such as phase/magnitude decoupling and long time window which is required to achieve sufficient frequency resolution. We propose Time-domain Audio Separation Network (TasNet) to overcome these limitations. We directly model the signal in the time-domain using an encoder-decoder framework and perform the source separation on nonnegative encoder outputs. This method removes the frequency decomposition step and reduces the separation problem to estimation of source masks on encoder outputs which is then synthesized by the decoder. Our system outperforms the current state-of-the-art causal and noncausal speech separation algorithms, reduces the computational cost of speech separation, and significantly reduces the minimum required latency of the output. This makes TasNet suitable for applications where low-power, real-time implementation is desirable such as in hearable and telecommunication devices.

Online Spectrogram Inversion for Low-Latency Audio Source Separation

“…We consider the update that does not modify the phase of s j (i.e., with a '+' sign in (12)), as it corresponds to the equality case of Eq. (7). Under such an update, ψ(s) = ψ + (s, y), which shows that ψ + is an auxiliary function for ψ.…”

Utterance-level Permutation Invariant Training with Latency-controlled BLSTM for Single-channel Multi-talker Speech Separation

“…The dataset is the same as the two-talker mixed dataset in [3], [4], [6], [7], [8], except that the sample rate is 16 kHz. It is generated by mixing the utterances in WSJ0 corpus at various signal-to-noise ratios uniformly chosen between 0 dB and 5 dB, and has 20k, 5k and 3k mixtures for training, validation and testing respectively.…”

“…Index Terms: multi-talker speech separation, permutation invariant training, latency-controlled BLSTM, speaker tracing I. INTRODUCTION Many advancements have been observed for monaural multi-talker speech separation [1], [2], [3], [4], [5], [6], [7], [8], [9], known as cocktail party problem [10], which is meaningful to many practical applications, such as humanmachine interaction, automatic meeting transcription etc. With the development of deep learning [11], a lot of innovations have been proposed, such as deep clustering [3], [4], deep attractor network [5], time-domain audio separation network [6], [9] and permutation invariant training (PIT) [7], [8].…”

End-to-End Training of Time Domain Audio Separation and Recognition

“…The topic of single-channel source separation has been examined extensively over the last few years, trying to solve the cocktail party problem with techniques such as Deep Clustering (DPCL) [3], Permutation Invariant Training (PIT) [4] and TasNet [5,6]. In DPCL, a neural network is trained to map each time-frequency bin to an embedding vector in a way that embedding vectors of the same speaker form a cluster in the embedding space.…”