Entropy-based pruning of hidden units to reduce DNN parameters

Mantena, Gautam Varma; Sim, Khe Chai

doi:10.1109/slt.2016.7846335

Cited by 3 publications

(1 citation statement)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…can be adjusted to control the rank of the effective offset matrix and the complexity of the adaptation model. Unlike other work on low-rank model compression [28,25,29], we apply low-rank approximation only to the domain-dependent parameters without compromising the performance of the base model.…”

Section: Factorized Hidden Layermentioning

confidence: 99%

Domain Adaptation Using Factorized Hidden Layer for Robust Automatic Speech Recognition

et al. 2018

Self Cite

View full text Add to dashboard Cite

Domain robustness is a challenging problem for automatic speech recognition (ASR). In this paper, we consider speech data collected for different applications as separate domains and investigate the robustness of acoustic models trained on multidomain data on unseen domains. Specifically, we use Factorized Hidden Layer (FHL) as a compact low-rank representation to adapt a multi-domain ASR system to unseen domains. Experimental results on two unseen domains show that FHL is a more effective adaptation method compared to selectively finetuning part of the network, without dramatically increasing the model parameters. Furthermore, we found that using singular value decomposition to initialize the low-rank bases of an FHL model leads to a faster convergence and improved performance.

show abstract

Section: Factorized Hidden Layermentioning

confidence: 99%