Learning Efficient Sparse Structures in Speech Recognition

Zhang, Jingchi; Wen, Wei; Deisher, Michael; Cheng, Hsin-Pai; Li, Hai; Chen, Yiran

doi:10.1109/icassp.2019.8683620

Cited by 3 publications

(2 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Wei et al [15] further applied group Lasso regularization on LSTMs and achieved 10.59× speedup without perplexity loss. Zhang et al [10] also extended the structural sparsity learning method to LSTM models for speech recognition and removed 72.5% parameters with negligible accuracy loss.…”

Section: Related Workmentioning

confidence: 99%

“…However, randomly distributed zeros in models do not have benefit for execution on hardware. [10] elaborates the benefit of structural sparsity over non-structural sparsity on locality and parallelism during hardware execution. To force zero parameters to form a regular arrangement, structural sparsity [11] is proposed for CNNs to learn sparse structures like channel and filter.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Structural Sparsification for Far-Field Speaker Recognition with Intel® Gna

Zhang

Huang

Deisher

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

Recently, deep neural networks (DNN) have been widely used in speaker recognition area. In order to achieve fast response time and high accuracy, the requirements for hardware resources increase rapidly. However, as the speaker recognition application is often implemented on mobile devices, it is necessary to maintain a low computational cost while keeping high accuracy in far-field condition. In this paper, we apply structural sparsification on time-delay neural networks (TDNN) to remove redundant structures and accelerate the execution. On our targeted hardware, our model can remove 60% of parameters and only slightly increasing equal error rate (EER) by 0.18% while our structural sparse model can achieve more than 2× speedup.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%