Asteroid: The PyTorch-Based Audio Source Separation Toolkit for Researchers

Pariente, Manuel; Cornell, Samuele; Cosentino, Joris; Sivasankaran, S.; Tzinis, Efthymios; Heitkaemper, Jens; Olvera, Michel; Stöter, Fabian-Robert; Hu, Mathieu; Martín-Doñas, Juan M.; Ditter, David; Frank, Ariel J.; Deleforge, Antoine; Vincent, Emmanuel

doi:10.21437/interspeech.2020-1673

Cited by 104 publications

(51 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All the models are stored in the same NVIDIA RTX8000-48GB GPU and we performed this analysis using the PyTorch profiler [32]. For Wavesplit we used the implementation in [33].…”

Section: Speed and Memory Comparisonmentioning

confidence: 99%

Attention Is All You Need In Speech Separation

Subakan

Ravanelli

Cornell

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

313

169

View full text Add to dashboard Cite

Recurrent Neural Networks (RNNs) have long been the dominant architecture in sequence-to-sequence learning. RNNs, however, are inherently sequential models that do not allow parallelization of their computations. Transformers are emerging as a natural alternative to standard RNNs, replacing recurrent computations with a multi-head attention mechanism.In this paper, we propose the SepFormer, a novel RNN-free Transformer-based neural network for speech separation. The Sep-Former learns short and long-term dependencies with a multi-scale approach that employs transformers. The proposed model achieves state-of-the-art (SOTA) performance on the standard WSJ0-2/3mix datasets. It reaches an SI-SNRi of 22.3 dB on WSJ0-2mix and an SI-SNRi of 19.5 dB on WSJ0-3mix. The SepFormer inherits the parallelization advantages of Transformers and achieves a competitive performance even when downsampling the encoded representation by a factor of 8. It is thus significantly faster and it is less memory-demanding than the latest speech separation systems with comparable performance.

show abstract

“…All the models are stored in the same NVIDIA RTX8000-48GB GPU and we performed this analysis using the PyTorch profiler [32]. For Wavesplit we used the implementation in [33].…”

Section: Speed and Memory Comparisonmentioning

confidence: 99%

Attention Is All You Need In Speech Separation

Subakan

Ravanelli

Cornell

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

313

169

View full text Add to dashboard Cite

show abstract

“…We experimentally evaluated the effectiveness of the SinkPIT based on a standard evaluation protocol provided in the Asteroid framework [18]. We basically used the default hyperparameters of the Asteroid, unless otherwise noted (see also the previous section).…”

Section: Methodsmentioning

confidence: 99%

“…Fig.2:The computation time to solve a permutation problem once. The "brute force" is based on the Asteroid[18] implementation.…”

mentioning

confidence: 99%

Towards Listening to 10 People Simultaneously: An Efficient Permutation Invariant Training of Audio Source Separation Using Sinkhorn’s Algorithm

Tachibana¹

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

In neural network-based monaural speech separation techniques, it has been recently common to evaluate the loss using the permutation invariant training (PIT) loss. However, the ordinary PIT requires to try all N ! permutations between N ground truths and N estimates. Since the factorial complexity explodes very rapidly as N increases, a PIT-based training works only when the number of source signals is small, such as N = 2 or 3. To overcome this limitation, this paper proposes a SinkPIT, a novel variant of the PIT losses, which is much more efficient than the ordinary PIT loss when N is large. The SinkPIT is based on Sinkhorn's matrix balancing algorithm, which efficiently finds a doubly stochastic matrix which approximates the best permutation in a differentiable manner. The author conducted an experiment to train a neural network model to decompose a single-channel mixture into 10 sources using the SinkPIT, and obtained promising results.

show abstract

“…In our experiments, the permutation-invariant scale-invariant sourceto-distortion ratio (SI-SDR) [18] is taken as the training objective and evaluation metric. All experiments are trained with the Adam optimizer for 100 epochs 1 and based on the Asteroid toolbox [19].…”

Section: Training and Evaluation Setupmentioning

confidence: 99%

Auditory Filterbanks Benefit Universal Sound Source Separation

Chen

Seeber

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

For separating two arbitrary sources from monaural recordings, the encoder-separator-decoder framework is popular in recent years. We investigated three kinds of filterbanks in the encoder: free, parameterized, and fixed. We proposed parameterized Gammatone and Gammachirp filterbanks, which improved performance with fewer parameters and better interpretability. Next, the properties of different filterbanks were investigated. Through training the network, an entirely freely learned filterbank emerges with properties similar to a series of bandpass filters spaced on a nonlinear scale -similar to the auditory system. We also explored the underlying separation mechanisms learned by the network through a classic auditory segregation experiment, revealing that the model separates mixtures based on the general principle (proximity of frequency and time). In summary, results demonstrate that the separation network automatically picks up the filterbank properties and separation mechanisms that are similar to those which have developed over millions of years in humans.

show abstract

Asteroid: The PyTorch-Based Audio Source Separation Toolkit for Researchers

Cited by 104 publications

References 36 publications

Attention Is All You Need In Speech Separation

Attention Is All You Need In Speech Separation

Towards Listening to 10 People Simultaneously: An Efficient Permutation Invariant Training of Audio Source Separation Using Sinkhorn’s Algorithm

Auditory Filterbanks Benefit Universal Sound Source Separation

Contact Info

Product

Resources

About