Interrupted and Cascaded Permutation Invariant Training for Speech Separation

Abstract

Permutation Invariant Training (PIT) has long been a stepping stone method for training speech separation model in handling the label ambiguity problem. With PIT selecting the minimum cost label assignments dynamically, very few studies considered the separation problem to be optimizing both the model parameters and the label assignments, but focused on searching for good model architecture and parameters. In this paper, we investigate instead for a given model architecture the various flexible label assignment strategies for training the model, rather than directly using PIT. Surprisingly, we discover a significant performance boost compared to PIT is possible if the model is trained with fixed label assignments and a good set of labels is chosen. With fixed label training cascaded between two sections of PIT, we achieved the stateof-the-art performance on WSJ0-2mix without changing the model architecture at all.

On Permutation Invariant Training For Speech Source Separation

“…Our SI-SNRi results (16.5 and 17.5 dB) are promising. Further, we also confirmed the reduction of permutation errors and generalization of improvements to other test sets, which was not tested in [7,8].…”

“…Prob-PIT [7] considers the probabilities of all utterance level permutations, rather than just the best one, improving the initial training stage when wrong alignments are likely to happen. A similar idea is employed by Yang et al [8], who trained a Conv-TasNet with uPIT and fixed alignments in turns, reporting 17.5 dB SI-SNRi. They also implemented Prob-PIT for Conv-TasNet and obtained 15.9 dB.…”

“…However, the permutation frequently changes over frames at inference time. Improvements to PIT roughly fall into two categories: (i) designing a permutation (or speaker) tracking algorithm for tPIT [2,4,5]; and (ii) designing better uPIT objectives to further strengthen permutation consistency [6][7][8][9]. Along these two lines, our work takes a close look at tPIT+clustering, a recent idea introduced by Deep CASA [2], that targets at accurate frame level separation (tPIT) and speaker tracking (clustering) in two stages.…”

Single-channel speech separation using Soft-minimum Permutation Invariant Training

“…This is the technique used in Permutation Invariant Training (PIT) speech separation method which has been shown to be effective in addressing the permutation ambiguity [35]. However, it has been discussed [39], [37] that the hard decision on choosing the minimum cost as the best solution results in training a sub-optimal separation model. To be more specific, the process of choosing the correct separation error is more challenging in the initial epochs of training, where the network is still naive and its outputs are not reliable.…”

Time-Domain Speech Extraction with Spatial Information and Multi Speaker Conditioning Mechanism

“…Although the PIT forces the frames belonging to the same speaker to be aligned with the same output stream, frames inside one utterance can still flip between different sources, leading to a poor separation performance. Alternatively, the initial PIT-based separation model can be further trained with a fixed label training strategy [3], or a long term dependency can be imposed to the output streams by adding an additional speaker identity loss [4,5]. Another issue in blind source separation is that the speaker order of the separated signals during inference is also unknown, and needs to be identified by a speaker recognition system.…”