DTDR–ALSTM: Extracting dynamic time-delays to reconstruct multivariate data for improving attention-based LSTM industrial time series prediction models

Li, Jince; Yang, Bo; Li, Hongguang; Wang, Yongjian; Chu, Qi; Liu, Yi

doi:10.1016/j.knosys.2020.106508

“…Not surprisingly attention based approaches are increasingly being applied to industry problems 17 . Li et al 18 present a novel approach to extracting dynamic time-delays to reconstruct multivariate data for an improved attention-based LSTM prediction model and apply it in the context of industrial distillation and methanol production processes. But they do not explicitly consider failure propagation in concatenated manufacturing systems to evaluate failure criticality and to generate a reliable failure impact prediction.…”

Section: /11mentioning

confidence: 99%

Identifying Cause-and-Effect Relationships of Manufacturing Errors using Sequence-to-Sequence Learning

Reimer¹,

Wang²,

Sofiane³

et al. 2022

Preprint

0

View full text Add to dashboard Cite

In car-body production the pre-formed sheet metal parts of the body are assembled on fully-automated production lines. The body passes through multiple stations in succession, and is processed according to the order requirements. The timely completion of orders depends on the individual station-based operations concluding within their scheduled cycle times. If an error occurs in one station, it can have a knock-on effect, resulting in delays on the downstream stations. To the best of our knowledge, there exist no methods for automatically distinguishing between source and knock-on errors in this setting, as well as establishing a causal relation between them. Utilizing real-time information about conditions collected by a production data acquisition system, we propose a novel vehicle manufacturing analysis system, which uses deep learning to establish a link between source and knock-on errors. We benchmark three sequence-to-sequence models, and introduce a novel composite time-weighted action metric for evaluating models in this context. We evaluate our framework on a real-world car production dataset recorded by Volkswagen Commercial Vehicles. Surprisingly we find that 71.68% of sequences contain either a source or knock-on error. With respect to seq2seq model training, we find that the Transformer demonstrates a better performance compared to LSTM and GRU in this domain, in particular when the prediction range with respect to the durations of future actions is increased.

show abstract

“…( 1234567890 Not surprisingly attention based approaches are increasingly being applied to industry problems 19 . Li et al 20 present a novel approach to extracting dynamic time-delays to reconstruct multivariate data for an improved attention-based LSTM prediction model and apply it in the context of industrial distillation and methanol production processes. But they do not explicitly consider failure propagation in concatenated manufacturing systems to evaluate failure criticality and to generate a reliable failure impact prediction.…”

Section: Related Workmentioning

confidence: 99%

Identifying cause-and-effect relationships of manufacturing errors using sequence-to-sequence learning

Reimer

¹

,

Wang

²

,

Sofiane

³

et al. 2022

Sci Rep

0

View full text Add to dashboard Cite

In car-body production the pre-formed sheet metal parts of the body are assembled on fully-automated production lines. The body passes through multiple stations in succession, and is processed according to the order requirements. The timely completion of orders depends on the individual station-based operations concluding within their scheduled cycle times. If an error occurs in one station, it can have a knock-on effect, resulting in delays on the downstream stations. To the best of our knowledge, there exist no methods for automatically distinguishing between source and knock-on errors in this setting, as well as establishing a causal relation between them. Utilizing real-time information about conditions collected by a production data acquisition system, we propose a novel vehicle manufacturing analysis system, which uses deep learning to establish a link between source and knock-on errors. We benchmark three sequence-to-sequence models, and introduce a novel composite time-weighted action metric for evaluating models in this context. We evaluate our framework on a real-world car production dataset recorded by Volkswagen Commercial Vehicles. Surprisingly we find that 71.68% of sequences contain either a source or knock-on error. With respect to seq2seq model training, we find that the Transformer demonstrates a better performance compared to LSTM and GRU in this domain, in particular when the prediction range with respect to the durations of future actions is increased.

show abstract

“…Statistical methods and machine learning algorithms can be used in this regard. In the literature in recent years, it is possible to see studies in which different machine learning algorithms such as artificial neural network (ANN), long short term memory (LSTM) and GA have been used in the aviation industry (He et al, 2022;Li et al, 2021aLi et al, , 2021bLiu et al, 2022a;Lyu et al, 2021). Some of these studies are presented in the following.…”

Section: Introductionmentioning

confidence: 99%

A new proposal for the prediction of an aircraft engine fuel consumption: a novel CNN-BiLSTM deep neural network model

Metlek

¹

2023

AEAT

View full text Add to dashboard Cite

Purpose The purpose of this study is to develop and test a new deep learning model to predict aircraft fuel consumption. For this purpose, real data obtained from different landings and take-offs were used. As a result, a new hybrid convolutional neural network (CNN)-bi-directional long short term memory (BiLSTM) model was developed as intended. Design/methodology/approach The data used are divided into training and testing according to the k-fold 5 value. In this study, 13 different parameters were used together as input parameters. Fuel consumption was used as the output parameter. Thus, the effect of many input parameters on fuel flow was modeled simultaneously using the deep learning method in this study. In addition, the developed hybrid model was compared with the existing deep learning models long short term memory (LSTM) and BiLSTM. Findings In this study, when tested with LSTM, one of the existing deep learning models, values of 0.9162, 6.476, and 5.76 were obtained for R2, root mean square error (RMSE), and mean absolute percentage error (MAPE), respectively. For the BiLSTM model when tested, values of 0.9471, 5.847 and 4.62 were obtained for R2, RMSE and MAPE, respectively. In the proposed hybrid model when tested, values of 0.9743, 2.539 and 1.62 were obtained for R2, RMSE and MAPE, respectively. The results obtained according to the LSTM and BiLSTM models are much closer to the actual fuel consumption values. The error of the models used was verified against the actual fuel flow reports, and an average absolute percent error value of less than 2% was obtained. Originality/value In this study, a new hybrid CNN-BiLSTM model is proposed. The proposed model is trained and tested with real flight data for fuel consumption estimation. As a result of the test, it is seen that it gives much better results than the LSTM and BiLSTM methods found in the literature. For this reason, it can be used in many different engine types and applications in different fields, especially the turboprop engine used in the study. Because it can be applied to different engines than the engine type used in the study, it can be easily integrated into many simulation models.

show abstract

DTDR–ALSTM: Extracting dynamic time-delays to reconstruct multivariate data for improving attention-based LSTM industrial time series prediction models

Cited by 51 publications

References 28 publications

Identifying Cause-and-Effect Relationships of Manufacturing Errors using Sequence-to-Sequence Learning

Identifying Cause-and-Effect Relationships of Manufacturing Errors using Sequence-to-Sequence Learning

Identifying cause-and-effect relationships of manufacturing errors using sequence-to-sequence learning

A new proposal for the prediction of an aircraft engine fuel consumption: a novel CNN-BiLSTM deep neural network model

Contact Info

Product

Resources

About