Towards Sequential Multivariate Fault Prediction for Vehicular Predictive Maintenance

Abstract

Predictive maintenance, which has traditionally used anomaly detection methods on sensory data, is now being replaced by event-based techniques. These methods utilise events with multiple temporal features, produced by diagnostic modules. This raises the need for predicting the next fault event in industrial machines, specially vehicles, that use Diagnostic Trouble Codes (DTCs). We propose a predictive maintenance approach, named Sequential Multivariate Fault Prediction (SMFP), for predicting the next multivariate DTC fault in an event sequence, using Long Short-Term Memory Networks (LSTMs) and jointly learned event embeddings. By performing an in-depth comparison of different architectural choices and contextual preprocessing techniques, we provide an initial baseline for SMFP that achieves top-3 accuracy of 63% on predicting multivariate fault with 3 collective output layers, using vehicle maintenance data as a case study.

DTCEncoder: A Swiss Army Knife Architecture for DTC Exploration, Prediction, Search and Model Interpretation

“…This model encodes a DTC sequence into a low-dimension representation, which encapsulates the sequential information that can be used for different downstream tasks. Along with enabling multiple functionalities, it surpasses the performance of SMFP [6].…”

“…In SMFP [6], the authors showed how LSTMs can predict the next DTC. A LSTM is a Recurrent Neural Network (RNN) where a recurrent loop acts as a memory to handle sequential dependencies.…”

“…The LSTM architecture used in SMFP [6] takes concatenated attribute embeddings A EM B and passes them through encoder LSTM layers. LSTM layers first calculate the new hidden state from the current input x t and the previous hidden state h t−1 .…”

“…Besides achieving the main objective of interpretability, the DTCEncoder also improved the previous baseline on the next event prediction task. We compared the performance of the DTCEncoder only against the LSTM architecture used in the SMFP approach [6], since, as far as we know, there is no other model that performs such task. We achieved the lowest validation loss of 4.36 and top-5 accuracy of 79% with our architecture, in contrast to validation loss of 4.52 and top-5 test accuracy of 76% with the architecture used by the SMFP model.…”

“…Deep learning models that consider sequential dependencies and solve the above-mentioned representation issues have also been proposed: in [5], a convolutional neural network and a gated recurrent network were used to learn embeddings for specific errors and their relationships; more in particular, the LSTM-based Sequential Multivariate Fault Prediction (SMFP) model [6] shows a promising direction for fault prediction 1 using a self-supervised end-to-end architecture. This task is illustrated in figure 1, where instead of providing the labels explicitly, the last event of each sequence is treated as the next target label.…”