Masked Autoencoder for Distribution Estimation on Small Structured Data Sets

Khajenezhad, Ahmad; Madani, Hatef; Beigy, Hamid

doi:10.1109/tnnls.2020.3026572

Cited by 5 publications

(3 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To meet the autoregressivity,

M^{A}

must be a strict lower‐diagonal matrix; otherwise, it needs to be filled with ones. Accordingly, the anomaly detection procedure of the MADE is displayed as below 36 …”

Section: Models and Methodsmentioning

confidence: 99%

“…Accordingly, the anomaly detection procedure of the MADE is displayed as below. 36 anomalyscore represents the anomaly score and is a discriminative index for anomaly detection. Here, the cross entropy estimated using the full-connection probability output by the model is taken as the anomaly score of samples.…”

Section: F I G U R E 3 Intelligent Maintenance Systems Of University ...mentioning

confidence: 99%

See 1 more Smart Citation

Anomaly detection in rolling bearings based on the Mel‐frequency cepstrum coefficient and masked autoencoder for distribution estimation

Xie

Liu

et al. 2022

Structural Contr & Hlth

View full text Add to dashboard Cite

It is difficult to establish a classification and recognition model of machinery and equipment based on labeled samples in the actual industrial environment because of the imperfect fault modes and data missing. To solve this problem, a semisupervised anomaly detection method based on masked autoencoders of distribution estimation (MADE) is designed. First, the Mel-frequency cepstrum coefficient (MFCC) is employed to extract fault features from vibration signals of rolling bearings. Then, a group of mask matrices are set on each hidden layer to overcome the perfect reconstruction problem of the autoencoders' input, and the full-connection probability of reconstruction is used to replace the reconstruction error and adopted as the anomaly score. Finally, the diagnostic threshold is determined according to the Youden index. Experimental results show that the MADE method can extract fault-sensitive features from a noisy industrial environment and introduce mask matrices renders to make the network autoregressive, thus solving the problem of perfect reconstruction of autoencoders. It is verified based on three rolling bearing datasets that the accuracy, precision, recall, and F1-score of the proposed method are confirmed to be all 100%. Moreover, the accuracy of the proposed method is 17.19% higher than that of the memory-inhibition method on the rolling bearing dataset provided by the Center for Intelligent Maintenance Systems (IMS) in University of Cincinnati (USA). The accuracy of the proposed method is also improved compared with other state-of-the-art anomaly detection methods.

show abstract

“…To meet the autoregressivity,

M^{A}

must be a strict lower‐diagonal matrix; otherwise, it needs to be filled with ones. Accordingly, the anomaly detection procedure of the MADE is displayed as below 36 …”

Section: Models and Methodsmentioning

confidence: 99%

Section: F I G U R E 3 Intelligent Maintenance Systems Of University ...mentioning

confidence: 99%

Anomaly detection in rolling bearings based on the Mel‐frequency cepstrum coefficient and masked autoencoder for distribution estimation

Xie

Liu

et al. 2022

Structural Contr & Hlth

View full text Add to dashboard Cite

show abstract

“…In addition, we use a shared Masked Autoencoder for Distribution Estimation (MADE) [19] in the output layer to model the sequential dependency between the edge probabilities of a given node. The MADE model is a well-known and efficient autoregressive model which has been an inspiration for other generative models, for instance, [20], [21]. MADE helps us to model the dependency of edge generation and efficiently generate edge probabilities all at once.…”

Section: Introductionmentioning

confidence: 99%

Gransformer: Transformer-based Graph Generation

Khajenezhad¹,

Osia²,

Karimian³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Transformers have become widely used in modern models for various tasks such as natural language processing and machine vision. This paper, proposes Gransformer, an algorithm for generating graphs that takes advantage of the transformer. We extend a simple autoregressive transformer encoder to exploit the structural information of the graph through efficient modifications. The attention mechanism is modified to consider the presence or absence of edges between each pair of nodes. We also introduce a graph-based familiarity measure that applies to both the attention and the positional coding. This autoregressive criterion, inspired by message passing algorithms, contains structural information about the graph. In the output layer, we also use a masked autoencoder for density estimation to efficiently model the generation of dependent edges. We also propose a technique to prevent the model from generating isolated nodes. We evaluate this method on two real-world datasets and compare it with some state-of-the-art autoregressive graph generation methods. Experimental results have shown that the proposed method performs comparative to these methods, including recurrent models and graph convolutional networks.

show abstract

Inferencing transportation mode using unsupervised deep learning approach exploiting GPS point-level characteristics

Dutta

Patra

2022

Appl Intell

View full text Add to dashboard Cite

Masked Autoencoder for Distribution Estimation on Small Structured Data Sets

Cited by 5 publications

References 4 publications

Anomaly detection in rolling bearings based on the Mel‐frequency cepstrum coefficient and masked autoencoder for distribution estimation

Anomaly detection in rolling bearings based on the Mel‐frequency cepstrum coefficient and masked autoencoder for distribution estimation

Gransformer: Transformer-based Graph Generation

Inferencing transportation mode using unsupervised deep learning approach exploiting GPS point-level characteristics

Contact Info

Product

Resources

About