Using An Attention-Based LSTM Encoder–Decoder Network for Near Real-Time Disturbance Detection

Yuan, Yuan; Lin, Lei; Huo, Lianzhi; Yun-long, Kong; Zhou, Zeng-Guang; Wu, Bin; Jia, Yan

doi:10.1109/jstars.2020.2988324

“…Especially designed to capture temporal dynamic behaviour, Recurrent Neural Networks (RNNs), in their various architectures such as Long Short-Term Memory (LSTMs) and Gated Recurrent Units (GRUs), have been extensively and successfully used for forecasting or detecting anomalies in multivariate time series data [21][22][23][24] . Bidirectional LSTMs were used to model genome data by 25 , whereas a combination of CNNs and LSTMs generates a model for epileptic seizure recognition using EEG signal analysis in 26 .…”

Section: Related Work and Contextmentioning

confidence: 99%

Learning the dynamics of realistic models of C. elegans nervous system with recurrent neural networks

Bărbulescu

¹

,

Mestre

²

,

Oliveira

³

et al. 2023

Sci Rep

View full text Add to dashboard Cite

Given the inherent complexity of the human nervous system, insight into the dynamics of brain activity can be gained from studying smaller and simpler organisms. While some of the potential target organisms are simple enough that their behavioural and structural biology might be well-known and understood, others might still lead to computationally intractable models that require extensive resources to simulate. Since such organisms are frequently only acting as proxies to further our understanding of underlying phenomena or functionality, often one is not interested in the detailed evolution of every single neuron in the system. Instead, it is sufficient to observe the subset of neurons that capture the effect that the profound nonlinearities of the neuronal system have in response to different stimuli. In this paper, we consider the well-known nematode Caenorhabditis elegans and seek to investigate the possibility of generating lower complexity models that capture the system’s dynamics with low error using only measured or simulated input-output information. Such models are often termed black-box models. We show how the nervous system of C. elegans can be modelled and simulated with data-driven models using different neural network architectures. Specifically, we target the use of state-of-the-art recurrent neural network architectures such as Long Short-Term Memory and Gated Recurrent Units and compare these architectures in terms of their properties and their accuracy (Root Mean Square Error), as well as the complexity of the resulting models. We show that Gated Recurrent Unit models with a hidden layer size of 4 are able to accurately reproduce the system response to very different stimuli. We furthermore explore the relative importance of their inputs as well as scalability to more scenarios.

show abstract

“…While Jiang et al 7 use time series multiple channel convolutional neural network integrated with the attention-based LSTM network for remaining useful life prediction of bearings. Near real-time disturbance detection becomes possible with the attention-based LSTM encoder-decoder network by Yuan et al 20 , which allows to align an input time series with the output time series and to dynamically choose the most relevant contextual information while forecasting. In contrast to previous work, we propose a workflow and evaluate seq2seq approaches for failure impact prediction in concatenated manufacturing systems.…”

Section: /11mentioning

confidence: 99%

Identifying Cause-and-Effect Relationships of Manufacturing Errors using Sequence-to-Sequence Learning

Reimer¹,

Wang²,

Sofiane³

et al. 2022

Preprint

0

View full text Add to dashboard Cite

In car-body production the pre-formed sheet metal parts of the body are assembled on fully-automated production lines. The body passes through multiple stations in succession, and is processed according to the order requirements. The timely completion of orders depends on the individual station-based operations concluding within their scheduled cycle times. If an error occurs in one station, it can have a knock-on effect, resulting in delays on the downstream stations. To the best of our knowledge, there exist no methods for automatically distinguishing between source and knock-on errors in this setting, as well as establishing a causal relation between them. Utilizing real-time information about conditions collected by a production data acquisition system, we propose a novel vehicle manufacturing analysis system, which uses deep learning to establish a link between source and knock-on errors. We benchmark three sequence-to-sequence models, and introduce a novel composite time-weighted action metric for evaluating models in this context. We evaluate our framework on a real-world car production dataset recorded by Volkswagen Commercial Vehicles. Surprisingly we find that 71.68% of sequences contain either a source or knock-on error. With respect to seq2seq model training, we find that the Transformer demonstrates a better performance compared to LSTM and GRU in this domain, in particular when the prediction range with respect to the durations of future actions is increased.

show abstract

“…While Jiang et al 9 use time series multiple channel convolutional neural network integrated with the attention-based LSTM network for remaining useful life prediction of bearings. Near real-time disturbance detection becomes possible with the attention-based LSTM encoder-decoder network by Yuan et al 22 , which allows to align an input time series with the output time series and to dynamically choose the most relevant contextual information while forecasting. In contrast to previous work, we propose a workflow and evaluate seq2seq approaches for failure impact prediction in concatenated manufacturing systems.…”

Section: Related Workmentioning

confidence: 99%

Identifying cause-and-effect relationships of manufacturing errors using sequence-to-sequence learning

Reimer

¹

,

Wang

²

,

Sofiane

³

et al. 2022

Sci Rep

0

View full text Add to dashboard Cite

In car-body production the pre-formed sheet metal parts of the body are assembled on fully-automated production lines. The body passes through multiple stations in succession, and is processed according to the order requirements. The timely completion of orders depends on the individual station-based operations concluding within their scheduled cycle times. If an error occurs in one station, it can have a knock-on effect, resulting in delays on the downstream stations. To the best of our knowledge, there exist no methods for automatically distinguishing between source and knock-on errors in this setting, as well as establishing a causal relation between them. Utilizing real-time information about conditions collected by a production data acquisition system, we propose a novel vehicle manufacturing analysis system, which uses deep learning to establish a link between source and knock-on errors. We benchmark three sequence-to-sequence models, and introduce a novel composite time-weighted action metric for evaluating models in this context. We evaluate our framework on a real-world car production dataset recorded by Volkswagen Commercial Vehicles. Surprisingly we find that 71.68% of sequences contain either a source or knock-on error. With respect to seq2seq model training, we find that the Transformer demonstrates a better performance compared to LSTM and GRU in this domain, in particular when the prediction range with respect to the durations of future actions is increased.

show abstract

Using An Attention-Based LSTM Encoder–Decoder Network for Near Real-Time Disturbance Detection

Cited by 26 publications

References 57 publications

Learning the dynamics of realistic models of C. elegans nervous system with recurrent neural networks

Learning the dynamics of realistic models of C. elegans nervous system with recurrent neural networks

Identifying Cause-and-Effect Relationships of Manufacturing Errors using Sequence-to-Sequence Learning

Identifying cause-and-effect relationships of manufacturing errors using sequence-to-sequence learning

Contact Info

Product

Resources

About