Understanding LSTM -- a tutorial into Long Short-Term Memory Recurrent Neural Networks

Staudemeyer, Ralf C.; Morris, Eric Rothstein

doi:10.48550/arxiv.1909.09586

Cited by 171 publications

(188 citation statements)

References 36 publications

Supporting

Mentioning

100

Contrasting

Unclassified

Order By: Relevance

“…They are composed of LSTM cells capable of capturing long-term dependencies in sequences while attenuate gradient vanishing/exploding problem [28]. This capacity is achieved by the use of forget and update gates to modify memory cell state that allow gradients to also flow unchanged [29,30]. The LSTM memory cells are composed by self-loops that encoded temporal information in the cell states, and three regulators gates that operate the flow of information within each cell.…”

Section: Long Short Term Memorymentioning

confidence: 99%

See 1 more Smart Citation

Water Flow Forecasting Based on River Tributaries Using Long Short-Term Memory Ensemble Model

et al. 2021

View full text Add to dashboard Cite

Water flow forecasts are an essential information for energy production, management and hydropower control. Advanced actions to optimize electricity production can be taken based on predicted information. This work proposes an ensemble strategy using recurrent neural networks to generate a forecast of water flow at Jirau Hydroelectric Power Plant (HPP), installed on the Madeira River in Brazil. The ensemble strategy consists of combining three long short-term memory (LSTM) networks that model the Madeira River and two of its tributaries: Mamoré and Abunã rivers. The historical data from streamflow of the Madeira river and its tributaries are used to validate the ensemble LSTM model, where each time series of river tributaries are modeled separated by LSTM models and the result used as input for another LSTM model in order to forecast the streamflow of the main river. The experimental results present low errors for training and test sets for individual LSTM networks and ensemble model. In addition, these results were compared with the operational forecasts performed by Jirau HPP. The proposed model showed better accuracy in four of the five scenarios tested, which indicates a promising approach to be explored in water flow forecasting based on river tributaries.

show abstract

Section: Long Short Term Memorymentioning

confidence: 99%

“…The three gates are called: forget gate f g , input gate i g and output gate o g , which operate the information flow by erasing, writing and reading, respectively. Therefore, LSTM models memorize information at different intervals and are suitable to predict time series with a certain duration interval [30,31].…”

Section: Long Short Term Memorymentioning

confidence: 99%

Water Flow Forecasting Based on River Tributaries Using Long Short-Term Memory Ensemble Model

et al. 2021

View full text Add to dashboard Cite

show abstract

“…jects. [14] regards the common occurrence frequency of object pairs as prior knowledge and utilizes LSTM (Long shortterm memory) [15] as an encoder to transfer context information to improve the feature representation between objects.…”

Section: -mentioning

confidence: 99%

DeepWORD: A GCN-Based Approach for Owner-Member Relationship Detection in Autonomous Driving

Wu¹,

Wang²,

Wang³

et al. 2021

2021 IEEE International Conference on Multimedia and Expo (ICME)

View full text Add to dashboard Cite

It's worth noting that the owner-member relationship between wheels and vehicles has an significant contribution to the 3D perception of vehicles, especially in the embedded environment. However, there are currently two main challenges about the above relationship prediction: i) The traditional heuristic methods based on IoU can hardly deal with the traffic jam scenarios for the occlusion. ii) It is difficult to establish an efficient applicable solution for the vehicle-mounted system. To address these issues, we propose an innovative relationship prediction method, namely DeepWORD, by designing a graph convolution network (GCN). Specifically, we utilize the feature maps with local correlation as the input of nodes to improve the information richness. Besides, we introduce the graph attention network (GAT) to dynamically amend the prior estimation deviation. Furthermore, we establish an annotated owner-member relationship dataset called WORD as a large-scale benchmark, which will be available soon. The experiments demonstrate that our solution achieves state-ofthe-art accuracy and real-time in practice.

show abstract

“…To train the CG agent we subdivide its architecture into a block which estimates the gradient, and into another block that controls an internal memory M (t i ) of the chemical field (i.e., the chemical memory control cell (CMC)). The latter is inspired by the well-known long short-term memory (LSTM) cell [39,40]. The first block is trained using the NEAT algorithm: it takes as input L T (t i ) and V T (t i ) as well as two recurrent variables C x (t i ) and G x (t i ) and maps this information onto a control output C y (t i ) and an estimated value of the instantaneous chemical gradient G y (t i ) ∈ [−1, 1], both to be forwarded to the CMC cell.…”

Section: Phase One: Learning Unidirectional Locomotionmentioning

confidence: 99%

Microswimmers learning chemotaxis with genetic algorithms

Hartl,

Hübl,

Kahl

et al. 2021

Preprint

View full text Add to dashboard Cite

Various microorganisms and some mammalian cells are able to swim in viscous fluids by performing nonreciprocal body deformations, such as rotating attached flagella or by distorting their entire body. In order to perform chemotaxis, i.e. to move towards and to stay at high concentrations of nutrients, they adapt their swimming gaits in a nontrivial manner. We propose a model how microswimmers are able to autonomously adapt their shape in order to swim towards high field concentrations using an internal decision making machinery modeled by an artificial neural network. We present two methods to measure chemical gradients, spatial and temporal sensing, as known for swimming mammalian cells and bacteria, respectively. Using the NEAT genetic algorithm surprisingly simple neural networks evolve which control the shape deformations of the microswimmer and allow them to navigate in static and complex time-dependent chemical environments. By introducing noisy signal transmission in the neural network the well-known biased run-and-tumble motion emerges. Our work demonstrates that the evolution of a simple internal decision-making machinery, which we can fully interpret and is coupled to the environment, allows navigation in diverse chemical landscapes. These findings are of relevance for intracellular biochemical sensing mechanisms of single cells, or for the simple nervous system of small multicellular organisms such as C. elegans.

show abstract

Understanding LSTM -- a tutorial into Long Short-Term Memory Recurrent Neural Networks

Cited by 171 publications

References 36 publications

Water Flow Forecasting Based on River Tributaries Using Long Short-Term Memory Ensemble Model

Water Flow Forecasting Based on River Tributaries Using Long Short-Term Memory Ensemble Model

DeepWORD: A GCN-Based Approach for Owner-Member Relationship Detection in Autonomous Driving

Microswimmers learning chemotaxis with genetic algorithms

Contact Info

Product

Resources

About