Period-aware content attention RNNs for time series forecasting with missing values

Cinar, Yagmur Gizem; Mirisaee, Hamid; Goswami, Parantapa; Gaussier, Éric; Aït-Bachir, Ali

doi:10.1016/j.neucom.2018.05.090

Cited by 90 publications

(54 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The attention mechanism in deep learning draws on the way of human's attention, and it has shown its effectiveness in lots of fields and applications, such as computer vision [33], natural language processing [34], [35], and prediction [36], [37]. Specifically, it is a weighted sum strategy that can automatically and adaptively analyze which part of data or information in the input is more significant.…”

Section: Attention Mechanismmentioning

confidence: 99%

CAME: Content- and Context-Aware Music Embedding for Recommendation

Wang

Zhang

et al. 2021

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Traditional recommendation methods suffer from limited performance, which can be addressed by incorporating abundant auxiliary/side information. This article focuses on a personalized music recommender system that incorporates rich content and context data in a unified and adaptive way to address the abovementioned problems. The content information includes music textual content, such as metadata, tags, and lyrics, and the context data incorporate users' behaviors, including music listening records, music playing sequences, and sessions. Specifically, a heterogeneous information network (HIN) is first presented to incorporate different kinds of content and context data. Then, a novel method called content-and context-aware music embedding (CAME) is proposed to obtain the low-dimension dense real-valued feature representations (embeddings) of music pieces from HIN. Especially, one music piece generally highlights different aspects when interacting with various neighbors, and it should have different representations separately. CAME seamlessly combines deep learning techniques, including convolutional neural networks and attention mechanisms, with the embedding model to capture the intrinsic features of music pieces as well as their dynamic relevance and interactions adaptively. Finally, we further infer users' general musical preferences as well as their contextual preferences for music and propose a content-and context-aware music recommendation method. Comprehensive experiments as well as quantitative and qualitative evaluations have been performed on real-world music data sets, and the results show that the proposed recommendation approach outperforms stateof-the-art baselines and is able to handle sparse data effectively.

show abstract

Section: Attention Mechanismmentioning

confidence: 99%

CAME: Content- and Context-Aware Music Embedding for Recommendation

Wang

Zhang

et al. 2021

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

show abstract

“…RNN is a type of feed-forward neural network. It is simply a network type that returns the value output from the neural network back to the input [20][21][22]. In this way, the output value of the last time point is transmitted back to the neural network, so that the weight calculation of each time point of the network model is related to the content of the previous time point, which means that the neural network is included.…”

Section: Recurrent Neural Network (Rnn)mentioning

confidence: 99%

Influenza-like illness prediction using a long short-term memory deep learning model with multiple open data sources

et al. 2020

View full text Add to dashboard Cite

The influenza problem has always been an important global issue. It not only affects people's health problems but is also an essential topic of governments and health care facilities. Early prediction and response is the most effective control method for flu epidemics. It can effectively predict the influenza-like illness morbidity, and provide reliable information to the relevant facilities. For social facilities, it is possible to strengthen epidemic prevention and care for highly sick groups. It can also be used as a reminder for the public. This study collects information on the influenzalike illness emergency department visits to the Taiwan Centers for Disease Control, and the PM 2.5 open-source data from the Taiwan Environmental Protection Administration's air quality monitoring network. By using deep learning techniques, the relevance of short-term estimates and the outbreak calculation method can be determined. The techniques are published by the WHO to determine whether the influenza-like illness situation is still in a stage of reasonable control. Finally, historical data and future forecasted data are integrated on the web page for visual presentation, to show the actual regional air quality situation and influenza-like illness data and to predict whether there is an outbreak of influenza in the region.

show abstract

“…Compared with LSTM, GRU has a faster convergence speed and no difference in accuracy. However, when the input time series is long, RNN series networks such as LSTM and GRU are prone to lose sequence information and it is difficult to model the structure information between data, which affects the accuracy of the model [32]. e attention mechanism is a resource allocation mechanism, which can assign different weights to input features so that the features containing important information will not disappear with the increase of step size, highlight the influence of more important information, and make the model easier to learn the long-distance interdependence in the sequence [33].…”

Section: Introductionmentioning

confidence: 99%

Early Fault Detection Method of Rolling Bearing Based on MCNN and GRU Network with an Attention Mechanism

Zhang

Cong

Yuan

et al. 2021

Shock and Vibration

View full text Add to dashboard Cite

Aiming at the problem of early fault diagnosis of rolling bearing, an early fault detection method of rolling bearing based on a multiscale convolutional neural network and gated recurrent unit network with attention mechanism (MCNN-AGRU) is proposed. This method first inputs multiple time scales rolling bearing vibration signals into the convolutional neural network to train the model through multiscale data processing and then adds the gated recurrent unit network with an attention mechanism to make the model predictive. Finally, the reconstruction error between the actual value and the predicted value is used to detect the early fault. The training data of this method is only normal data. The early fault detection in the operating condition monitoring and performance degradation assessment of the rolling bearing is effectively solved. It uses a multiscale data processing method to make the features extracted by CNN more robust and uses a GRU network with an attention mechanism to make the predictive ability of this method not affected by the length of the data. Experimental results show that the MCNN-AGRU rolling bearing early fault diagnosis method proposed in this paper can effectively detect the early fault of the rolling bearing and can effectively identify the type of rolling bearing fault.

show abstract

Period-aware content attention RNNs for time series forecasting with missing values

Cited by 90 publications

References 16 publications

CAME: Content- and Context-Aware Music Embedding for Recommendation

CAME: Content- and Context-Aware Music Embedding for Recommendation

Influenza-like illness prediction using a long short-term memory deep learning model with multiple open data sources

Early Fault Detection Method of Rolling Bearing Based on MCNN and GRU Network with an Attention Mechanism

Contact Info

Product

Resources

About