Tree Memory Networks for modelling long-term temporal dependencies

Abstract

In the domain of sequence modelling, Recurrent Neural Networks (RNN) have been capable of achieving impressive results in a variety of application areas including visual question answering, part-of-speech tagging and machine translation. However this success in modelling short term dependencies has not successfully transitioned to application areas such as trajectory prediction, which require capturing both short term and long term relationships. In this paper, we propose a Tree Memory Network (TMN) for jointly modelling both long term relationships between multiple sequences and short term relationships within a sequence, in sequence-to-sequence mapping problems. The proposed network architecture is composed of an input module, controller and a memory module. In contrast to related literature which models the memory as a sequence of historical states, we model the memory as a recursive tree structure. This structure more effectively captures temporal dependencies across both short and long term time periods through its hierarchical structure. We demonstrate the effectiveness and flexibility of the proposed TMN in two practical problems: aircraft trajectory modelling and pedestrian trajectory modelling in a surveillance setting. In both cases the proposed approach outperforms the current state-of-the-art. Furthermore, we perform an in depth analysis on the evolution of the memory module content over time and provide visual evidence on how the proposed TMN is able to map both short and long term relationships efficiently via a hierarchical structure.

Neural memory plasticity for medical anomaly detection

“…The authors in [1,5,3,2] and our prior works [6,20,21,4] have extensively demonstrated the effectiveness of what are termed "memory modules"…”

“…These dependencies are missed by models such as LSTMs and Gated Recurrent Units (GRUs) as they consider dependencies only within a given input sequence. Due to this ability external memory modules have gained traction in numerous domains, including language modelling [1], visual question answering [3], trajectory prediction [6,20], object tracking [5], and saliency modelling [4].…”

“…Neural Memory Networks (NMNs) have recently achieved tremendous success on larger knowledge bases via the use of an external memory to explicitly store and retrieve relevant information [1,2,3,4,5,6]. They elevate the temporal model capability of Recurrent Neural Networks (RNNs) [7] to capture both long-term and short-term relationships where the added capacity is utilised to store the information that constitutes these relationships.…”

GNER: A Generative Model for Geological Named Entity Recognition Without Labeled Data Using Deep Learning

“…The generative model then randomly chooses among the seeds to generate new sentences using an attention-based bidirectional long short-term memory (Bi-LSTM) model (Fernando et al, 2018;Wang et al, 2019). This approach dynamically adapts to extract and identify unseen instances.…”

“…To train the generative model, the method takes as input a collection of domain-specific entities and domain-general words along with their corresponding frequencies (referred to as sample seeds). The generative model then randomly chooses among the seeds to generate new sentences using an attention-based bidirectional long short-term memory (Bi-LSTM) model (Fernando et al, 2018;Wang et al, 2019). The process of generating sentences based on a unigram language model is iteratively repeated until the algorithm converges (Guo et al, 2017;Qiu et al, 2018;Quijano-Sánchez et al, 2018).…”