Time-Aware and Feature Similarity Self-Attention in Vessel Fuel Consumption Prediction

Abstract

An accurate vessel fuel consumption prediction is essential for constructing a ship route network and vessel management, leading to efficient sailings. Besides, ship data from monitoring and sensing systems accelerate fuel consumption prediction research. However, the ship data consist of three properties: sequential, irregular time interval, and feature importance, making the predicting problem challenging. In this paper, we propose Time-aware Attention (TA) and Feature-similarity Attention (FA) applied to bi-directional Long Short-Term Memory (LSTM). TA acquires time importance by nonlinear function from irregular time intervals in each sequence and emphasizes data depending on the importance. FA emphasizes data based on similarities of features in the sequence by estimating feature importance with learnable parameters. Finally, we propose the ensemble model of TA and FA-based BiLSTM. The ensemble model, which consists of fully connected layers, is capable of simultaneously capturing different properties of ship data. The experimental results on ship data showed that the proposed model improves the performance in predicting fuel consumption. In addition to model performance, visualization results of attention maps and feature importance help to understand data properties and model characteristics.

CRFormer: Complementary Reliability Perspective Transformer for Automotive Components Reliability Prediction Based on Claim Data

“…In the time-series prediction domain, [37]- [39] demonstrated that Transformer achieved better performance than the previous time-series models. [39] suggested that Transformer models achieved higher performance than LSTM models in vessel fuel consumption prediction and [37], [38] proved the outstanding of Transformer by comparing their methods with both statistical methods and RNNs in influenza and multi-horizon time-series prediction respectively. As previous studies have proven the success of Transformer in time-series prediction, researchers have proposed Transformer-based time-series models that satisfy both efficiency and performance [40]- [43].…”

“…Owing to the added advantage of an attention mechanism that alleviates the vanishing gradient problem, the sequential feature extraction ability of Transformer helps achieve state-of-theart performance in diverse domains such as pre-trained language models [32], image-relevant tasks [33]- [35], and a multi-modal task [36]. In the time-series prediction domain, [37]- [39] demonstrated that Transformer achieved better performance than the previous time-series models. [39] suggested that Transformer models achieved higher performance than LSTM models in vessel fuel consumption prediction and [37], [38] proved the outstanding of Transformer by comparing their methods with both statistical methods and RNNs in influenza and multi-horizon time-series prediction respectively.…”