State initialization for recurrent neural network modeling of time-series data

Mohajerin, Nima; Waslander, Steven L.

doi:10.1109/ijcnn.2017.7966138

Cited by 22 publications

(12 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In other words, at time t = 0 the recurrent layer can use a valid input (x 0 ) and a valid previous hidden state (h −1 ) that should reduce reconstruction error of the next frames. We train the additional parameters of this sub-network jointly with the rest of the system, without any modi cation to the loss function, as opposed to Mohajerin and Waslander [2017] who design a more complex hidden state initialization procedure that requires a separate loss function and a sub-sequence of timesteps as inputs for the state initializer.…”

Section: Hidden State Initializermentioning

confidence: 99%

Recurrent transition networks for character locomotion

Harvey

Pal²

2018

SIGGRAPH Asia 2018 Technical Briefs

View full text Add to dashboard Cite

Fig. 1. A transition animation (green) generated with our proposed system, from past and future contexts (magenta). Every fi h frame is shown.Manually authoring transition animations for a complete locomotion system can be a tedious and time-consuming task, especially for large games that allow complex and constrained locomotion movements, where the number of transitions grows exponentially with the number of states. In this paper, we present a novel approach, based on deep recurrent neural networks, to automatically generate such transitions given a past context of a few frames and a target character state to reach. We present the Recurrent Transition Network (RTN), based on a modi ed version of the Long-Short-Term-Memory (LSTM) network, designed speci cally for transition generation and trained without any gait, phase, contact or action labels. We further propose a simple yet principled way to initialize the hidden states of the LSTM layer for a given sequence which improves the performance and generalization to new motions. We both quantitatively and qualitatively evaluate our system and show that making the network terrain-aware by adding a local terrain representation to the input yields be er performance for rough-terrain navigation on long transitions. Our system produces realistic and uid transitions that rival the quality of Motion Capture-based ground-truth motions, even before applying any inverse-kinematics postprocess (video 1 ). Direct bene ts of our approach could be to accelerate the creation of transition variations for large coverage, or even to entirely replace transition nodes in an animation graph. We further explore applications of this model in a animation superresolution se ing where we temporally decompress animations saved at 1 frame per second and show that the network is able to reconstruct motions that are hard to distinguish from un-compressed locomotion sequences.

show abstract

Section: Hidden State Initializermentioning

confidence: 99%

Recurrent transition networks for character locomotion

Harvey

Pal²

2018

SIGGRAPH Asia 2018 Technical Briefs

View full text Add to dashboard Cite

show abstract

“…The network initializes the hidden and cell states to zero for the first cell. Depending on the learning task, other initialization mechanisms can be adopted for improving the learning performance or accelerating the training process [21].…”

Section: Network Architecturementioning

confidence: 99%

Multi-Perspective LSTM for Joint Visual Representation Learning

Sepas-Moghaddam¹,

Pereira²,

Correia³

et al. 2021

Preprint

View full text Add to dashboard Cite

We present a novel LSTM cell architecture capable of learning both intra-and inter-perspective relationships available in visual sequences captured from multiple perspectives. Our architecture adopts a novel recurrent joint learning strategy that uses additional gates and memories at the cell level. We demonstrate that by using the proposed cell to create a network, more effective and richer visual representations are learned for recognition tasks. We validate the performance of our proposed architecture in the context of two multi-perspective visual recognition tasks namely lip reading and face recognition. Three relevant datasets are considered and the results are compared against fusion strategies, other existing multi-input LSTM architectures, and alternative recognition solutions. The experiments show the superior performance of our solution over the considered benchmarks, both in terms of recognition accuracy and complexity. We make our code publicly available at https://github.com/arsm/MPLSTM.

show abstract

“…During the first phase, where k = 1, ..., τ I , the observed OGMs are given to the model. The first phase is mainly intended for RNN state initialization [16], and therefore, we will refer to it as the initialization phase, or init-phase for short. The second phase, namely the prediction phase, starts at k = τ + 1 and is intended for multi-step prediction, as the input OGMs are blank.…”

Section: Ogm Prediction In Autonomous Drivingmentioning

confidence: 99%

Multi-Step Prediction of Occupancy Grid Maps With Recurrent Neural Networks

Mohajerin

Rohani

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

View full text Add to dashboard Cite

We investigate the multi-step prediction of the drivable space, represented by Occupancy Grid Maps (OGMs), for autonomous vehicles. Our motivation is that accurate multi-step prediction of the drivable space can efficiently improve path planning and navigation resulting in safe, comfortable and optimum paths in autonomous driving. We train a variety of Recurrent Neural Network (RNN) based architectures on the OGM sequences from the KITTI dataset. The results demonstrate significant improvement of the prediction accuracy using our proposed difference learning method, incorporating motion related features, over the state of the art. We remove the egomotion from the OGM sequences by transforming them into a common frame. Although in the transformed sequences the KITTI dataset is heavily biased toward static objects, by learning the difference between consecutive OGMs, our proposed method provides accurate prediction over both the static and moving objects. A video of the performance of our method on the KITTI dataset is available at https: //youtu.be/Bskd0Z7eLFE.

show abstract

State initialization for recurrent neural network modeling of time-series data

Cited by 22 publications

References 17 publications

Recurrent transition networks for character locomotion

Recurrent transition networks for character locomotion

Multi-Perspective LSTM for Joint Visual Representation Learning

Multi-Step Prediction of Occupancy Grid Maps With Recurrent Neural Networks

Contact Info

Product

Resources

About