Maxout neurons for deep convolutional and LSTM neural networks in speech recognition

Cai, Meng; Liu, Jia

doi:10.1016/j.specom.2015.12.003

Cited by 83 publications

(38 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These improvements over the traditional CNN and RNN can help to learn more precise time domain or frequency domain information. For example, in [21], the convolutional maxout neural networks (CMNN) and recurrent maxout neural networks (RMNN) use local spectral properties within frames and long-term dependencies among frames. In our experiments, we compare the results achieved by our models to these already strong baselines of these other models.…”

Section: Experimental Results For Different Modelsmentioning

confidence: 99%

“…The solutions we propose include local window BLSTM, gated recurrent units, and residual architecture-based models. This work expands on our previous work [18][19][20][21]. Experiments are carried out on the Babel benchmark datasets, for low resource keyword search evaluations.…”

Section: Introductionmentioning

confidence: 94%

“…In order to achieve better performance, we select models using multilingual bottleneck features as inputs, then train them using the SMBR sequence training algorithm [17]. For fair comparison, we only show the cross-entropy WER for fbank input models, as in [21].…”

Section: Baseline Setupmentioning

confidence: 99%

See 2 more Smart Citations

Advanced recurrent network-based hybrid acoustic models for low resource speech recognition

Kang

Zhang

Liu

et al. 2018

J AUDIO SPEECH MUSIC PROC.

Self Cite

View full text Add to dashboard Cite

Recurrent neural networks (RNNs) have shown an ability to model temporal dependencies. However, the problem of exploding or vanishing gradients has limited their application. In recent years, long short-term memory RNNs (LSTM RNNs) have been proposed to solve this problem and have achieved excellent results. Bidirectional LSTM (BLSTM), which uses both preceding and following context, has shown particularly good performance. However, the computational requirements of BLSTM approaches are quite heavy, even when implemented efficiently with GPU-based high performance computers. In addition, because the output of LSTM units is bounded, there is often still a vanishing gradient issue over multiple layers. The large size of LSTM networks makes them susceptible to overfitting problems. In this work, we combine local bidirectional architecture, a new recurrent unit, gated recurrent units (GRU), and residual architectures to address the above problems. Experiments are conducted on the benchmark datasets released under the IARPA Babel Program. The proposed models achieve 3 to 10% relative improvements over their corresponding DNN or LSTM baselines across seven language collections. In addition, the new models accelerate learning speed by a factor of more than 1.6 compared to conventional BLSTM models. By using these approaches, we achieve good results in the IARPA Babel Program.

show abstract

Section: Experimental Results For Different Modelsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 94%

See 1 more Smart Citation

Advanced recurrent network-based hybrid acoustic models for low resource speech recognition

Kang

Zhang

Liu

et al. 2018

J AUDIO SPEECH MUSIC PROC.

Self Cite

View full text Add to dashboard Cite

show abstract

“…e long-short-term memory neural network (LSTM) has memory cells, which can extract deep features from a small number of samples. It is suitable for processing time series and has achieved sound application effects in many fields [18,19]. However, the parameters of the LSTM model are usually determined by experience, so the subjectivity is strong and will affect the fitting ability of the model.…”

Section: Introductionmentioning

confidence: 99%

Nickel Price Forecast Based on the LSTM Neural Network Optimized by the Improved PSO Algorithm

Shao

Zhao

et al. 2019

Mathematical Problems in Engineering

View full text Add to dashboard Cite

Nickel is a vital strategic metal resource with commodity and financial attributes simultaneously, whose price fluctuation will affect the decision-making of stakeholders. Therefore, an effective trend forecast of nickel price is of great reference for the risk management of the nickel market’s participants; yet, traditional forecast methods are defective in prediction accuracy and applicability. Therefore, a prediction model of nickel metal price is proposed based on improved particle swarm optimization algorithm (PSO) combined with long-short-term memory (LSTM) neural networks, for higher reliability. This article introduces a nonlinear decreasing assignment method and sine function to improve the inertia weight and learning factor of PSO, respectively, and then uses the improved PSO algorithm to optimize the parameters of LSTM. Nickel metal’s closing prices in London Metal Exchange are sampled for empirical analysis, and the improved PSO-LSTM model is compared with the conventional LSTM and the integrated moving average autoregressive model (ARIMA). The results show that compared with the standard PSO, the improved PSO has a faster convergence rate and can improve the prediction accuracy of the LSTM model effectively. In addition, compared with the conventional LSTM model and the integrated moving average autoregressive (ARIMA) model, the prediction error of the LSTM model optimized by the improved PSO is reduced by 9% and 13%, respectively, which has high reliability and can provide valuable guidance for relevant managers.

show abstract

“…Taking the classical BP neural network as an example, obtaining highprecision features becomes more difficult while the layers are few. If the number of layers is excessive, then the gradient may disappear, and the local optimal solution is another defect that is difficult to overcome [9].…”

Section: Introductionmentioning

confidence: 99%

Deep Sparse Autoencoder for Feature Extraction and Diagnosis of Locomotive Adhesion Status

Zhang

Cheng

Liu

et al. 2018

Journal of Control Science and Engineering

View full text Add to dashboard Cite

The model is difficult to establish because the principle of the locomotive adhesion process is complex. This paper presents a data-driven adhesion status fault diagnosis method based on deep learning theory. The adhesion coefficient and creep speed of a locomotive constitute the characteristic vector. The sparse autoencoder unsupervised learning network studies the input vector, and the single-layer network is superimposed to form a deep neural network. Finally, a small amount of labeled data is used to fine-tune training the entire deep neural network, and the locomotive adhesion state fault diagnosis model is established. Experimental results show that the proposed method can achieve a 99.3% locomotive adhesion state diagnosis accuracy and satisfy actual engineering monitoring requirements.

show abstract

Maxout neurons for deep convolutional and LSTM neural networks in speech recognition

Cited by 83 publications

References 10 publications

Advanced recurrent network-based hybrid acoustic models for low resource speech recognition

Advanced recurrent network-based hybrid acoustic models for low resource speech recognition

Nickel Price Forecast Based on the LSTM Neural Network Optimized by the Improved PSO Algorithm

Deep Sparse Autoencoder for Feature Extraction and Diagnosis of Locomotive Adhesion Status

Contact Info

Product

Resources

About