Deep long short-term memory networks for speech recognition

Chien, Jen‐Tzung; Misbullah, Alim

doi:10.1109/iscslp.2016.7918375

Cited by 9 publications

(7 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sound event prediction using convolutional RNNs [195]. Audio tagging using Convolutional GRUs [196]. Early heart failure detection is proposed using RNNs [197].…”

Section: Rnn Applicationsmentioning

confidence: 99%

A State-of-the-Art Survey on Deep Learning Theory and Architectures

et al. 2019

View full text Add to dashboard Cite

In recent years, deep learning has garnered tremendous success in a variety of application domains. This new field of machine learning has been growing rapidly and has been applied to most traditional application domains, as well as some new areas that present more opportunities. Different methods have been proposed based on different categories of learning, including supervised, semi-supervised, and un-supervised learning. Experimental results show state-of-the-art performance using deep learning when compared to traditional machine learning approaches in the fields of image processing, computer vision, speech recognition, machine translation, art, medical imaging, medical information processing, robotics and control, bioinformatics, natural language processing, cybersecurity, and many others. This survey presents a brief survey on the advances that have occurred in the area of Deep Learning (DL), starting with the Deep Neural Network (DNN). The survey goes on to cover Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), including Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU), Auto-Encoder (AE), Deep Belief Network (DBN), Generative Adversarial Network (GAN), and Deep Reinforcement Learning (DRL). Additionally, we have discussed recent developments, such as advanced variant DL techniques based on these DL approaches. This work considers most of the papers published after 2012 from when the history of deep learning began. Furthermore, DL approaches that have been explored and evaluated in different application domains are also included in this survey. We also included recently developed frameworks, SDKs, and benchmark datasets that are used for implementing and evaluating deep learning approaches. There are some surveys that have been published on DL using neural networks and a survey on Reinforcement Learning (RL). However, those papers have not discussed individual advanced techniques for training large-scale deep learning models and the recently developed method of generative models.

show abstract

“…Sound event prediction using convolutional RNNs [195]. Audio tagging using Convolutional GRUs [196]. Early heart failure detection is proposed using RNNs [197].…”

Section: Rnn Applicationsmentioning

confidence: 99%

A State-of-the-Art Survey on Deep Learning Theory and Architectures

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Each hidden layer has 1024 units. LSTM3 means 3 stacks of LSTM where each stack contains 1024 cells [22]. Experimental setup of CNN is referred to [10].…”

Section: Methodsmentioning

confidence: 99%

Deep Neural Factorization for Speech Recognition

Chien¹,

Shen²

2017

Interspeech 2017

Self Cite

View full text Add to dashboard Cite

Conventional speech recognition system is constructed by unfolding the spectral-temporal input matrices into one-way vectors and using these vectors to estimate the affine parameters of neural network according to the vector-based error backpropagation algorithm. System performance is constrained because the contextual correlations in frequency and time horizons are disregarded and the spectral and temporal factors are excluded. This paper proposes a spectral-temporal factorized neural network (STFNN) to tackle this weakness. The spectral-temporal structure is preserved and factorized in hidden layers through two ways of factor matrices which are trained by using the factorized error backpropagation. Affine transformation in standard neural network is generalized to the spectro-temporal factorization in STFNN. The structural features or patterns are extracted and forwarded towards the softmax outputs. A deep neural factorization is built by cascading a number of factorization layers with fully-connected layers for speech recognition. An orthogonal constraint is imposed in factor matrices for redundancy reduction. Experimental results show the merit of integrating the factorized features in deep feedforward and recurrent neural networks for speech recognition.

show abstract

“…Other speech recognition studies using LSTM network have shown significant performance improvement compared to previous state-of-the-art DBN based models. Furthermore, Chien et al [88] performed an extensive experiment with various LSTM architectures for speech recognition and compared the performance with state-of-the-art models. To summarize key results from DBNs and RNNs (including LSTMs), Table III shows different problems and error rates achieved by the state-of-the-art speech recognition models.…”

Section: Deep Learning In Speech Recognitionmentioning

confidence: 99%

“…RNN models are leading the current speech recognition systems, especially in the emerging applications of NLP. Several revolutionary variants of RNN such as the non-linear structure of LSTM [34,88] and the hybrid CNN-LSTM architecture [156] have made substantial improvements in the field of intelligent speech recognition and automatic image captioning.…”

Section: Summary Of Surveymentioning

confidence: 99%