On-demand compute reduction with stochastic wav2vec 2.0

Vyas, Apoorv; Hsu, Wei-Ning; Auli, Michael; Baevski, Alexei

doi:10.21437/interspeech.2022-10584

Cited by 6 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Downstream results. First, the reported results in Table 1 are in-line with previous works evaluating partially pre-trained wav2vec 2.0 [18]. Speaker verification EER, for instance, are even better than those reported within the official SUPERB leaderboard with 6.0% against 4.1% in our experiments for the base wav2vec 2.0.…”

Section: Downstream Evaluationsupporting

confidence: 89%

See 1 more Smart Citation

On the (In)Efficiency of Acoustic Feature Extractors for Self-Supervised Speech Representation Learning

Parcollet¹,

Zhang²,

Rogier³

et al. 2023

Interspeech 2023

View full text Add to dashboard Cite

Speech representations learned with self-supervised learning (SSL) have the potential to significantly improve the performance of a number of audio applications, especially when availability of labeled data from the deployment domain is limited. Despite their successes, SSL training methods are compute-and memory-heavy, and require large investments in computing infrastructure, thus putting it out of the reach of most institutions. Therefore, building efficient model architectures is essential for the wide-scale adoption of SSL in speech technologies. CNN-based Acoustic Feature Extractors (AFE), which are widely used as encoders of acoustic waveforms, remain one of the main efficiency bottlenecks. This work proposes replacing CNN-based AFEs with more efficient ones and demonstrates that SSL pre-training time and memory consumption can be reduced by a factor of two to three over existing methods while preserving performances in speech-, command-, and speakerrecognition tasks.

show abstract

Section: Downstream Evaluationsupporting

confidence: 89%

“…SOTA SSL-trained models rely on the same CNN-based AFE combined with a large vanilla Transformer. Careful engineering of these two parts could lead to significant efficiency gains [18] and allow pre-training on mid-tier GPU (e.g., Nvidia Ti 80/90 families) [16].…”

Section: Introductionmentioning

confidence: 99%