From Feedforward to Recurrent LSTM Neural Networks for Language Modeling

Sundermeyer, Martin; Ney, Hermann; Schlüter, Ralf

doi:10.1109/taslp.2015.2400218

Cited by 779 publications

(678 citation statements)

References 29 publications

Supporting

Mentioning

662

Contrasting

Unclassified

Order By: Relevance

“…RNN has a strong power to handle a sequence with temporal correlation and it has been widely utilized in solving time-varying problems [42,43], especially natural language processing [44]. Unlike a common neural network that has no connections within hidden layers, RNN is able to connect hidden layers with the former ones circularly.…”

Section: Recurrent Neural Network (Rnn)mentioning

confidence: 99%

Ensemble Recurrent Neural Network Based Probabilistic Wind Speed Forecasting Approach

et al. 2018

View full text Add to dashboard Cite

Abstract:Wind energy is a commonly utilized renewable energy source, due to its merits of extensive distribution and rich reserves. However, as wind speed fluctuates violently and uncertainly at all times, wind power integration may affect the security and stability of power system. In this study, we propose an ensemble model for probabilistic wind speed forecasting. It consists of wavelet threshold denoising (WTD), recurrent neural network (RNN) and adaptive neuro fuzzy inference system (ANFIS). Firstly, WTD smooths the wind speed series in order to better capture its variation trend. Secondly, RNNs with different architectures are trained on the denoising datasets, operating as sub-models for point wind speed forecasting. Thirdly, ANFIS is innovatively established as the top layer of the entire ensemble model to compute the final point prediction result, in order to take full advantages of a limited number of deep-learning-based sub-models. Lastly, variances are obtained from sub-models and then prediction intervals of probabilistic forecasting can be calculated, where the variances inventively consist of modeling and forecasting uncertainties. The proposed ensemble model is established and verified on less than one-hour-ahead ultra-short-term wind speed forecasting. We compare it with other soft computing models. The results indicate the feasibility and superiority of the proposed model in both point and probabilistic wind speed forecasting.

show abstract

Section: Recurrent Neural Network (Rnn)mentioning

confidence: 99%

Ensemble Recurrent Neural Network Based Probabilistic Wind Speed Forecasting Approach

et al. 2018

View full text Add to dashboard Cite

show abstract

“…An LSTM network contains LSTM units in RNN and an LSTM unit is a recurrent network unit that excels at remembering values for either long or short durations of time (Graves, 2012b;Sundermeyer et al, 2012). It contains an input gate, a forget gate, an output gate and a memory cell.…”

Section: Lstm Networkmentioning

confidence: 99%

Towards Automatic Generation of Product Reviews from Aspect-Sentiment Scores

Zang¹,

Wan²

2017

Proceedings of the 10th International Conference on Natural Language Generation

View full text Add to dashboard Cite

Data-to-text generation is very essential and important in machine writing applications. The recent deep learning models, like Recurrent Neural Networks (RNNs), have shown a bright future for relevant text generation tasks. However, rare work has been done for automatic generation of long reviews from user opinions. In this paper, we introduce a deep neural network model to generate long Chinese reviews from aspect-sentiment scores representing users' opinions. We conduct our study within the framework of encoderdecoder networks, and we propose a hierarchical structure with aligned attention in the Long-Short Term Memory (LSTM) decoder. Experiments show that our model outperforms retrieval based baseline methods, and also beats the sequential generation models in qualitative evaluations.

show abstract

“…We chose for our experiments the Long-short Term Memory (LSTM) [4] networks architecture due its better ability of learning [14].The scheme of one LSTM cell is shown in Figure 2. The simpler "vanilla" RNN network showed to be unstable while we were changing the input of the model in experiments.…”

Section: Our Model Architecturementioning

confidence: 99%

On Continuous Space Word Representations as Input of LSTM Language Model

Soutner

Müller

2015

Statistical Language and Speech Processing

View full text Add to dashboard Cite

Abstract. Artificial neural networks have become the state-of-the-art in the task of language modelling whereas Long-Short Term Memory (LSTM) networks seem to be an efficient architecture. The continuous skip-gram and the continuous bag of words (CBOW) are algorithms for learning quality distributed vector representations that are able to capture a large number of syntactic and semantic word relationships. In this paper, we carried out experiments with a combination of these powerful models: the continuous representations of words trained with skipgram/CBOW /GloVe method, word cache expressed as a vector using latent Dirichlet allocation (LDA). These all are used on the input of LSTM network instead of 1-of-N coding traditionally used in language models. The proposed models are tested on Penn Treebank and MALACH corpus.

show abstract

From Feedforward to Recurrent LSTM Neural Networks for Language Modeling

Cited by 779 publications

References 29 publications

Ensemble Recurrent Neural Network Based Probabilistic Wind Speed Forecasting Approach

Ensemble Recurrent Neural Network Based Probabilistic Wind Speed Forecasting Approach

Towards Automatic Generation of Product Reviews from Aspect-Sentiment Scores

On Continuous Space Word Representations as Input of LSTM Language Model

Contact Info

Product

Resources

About