Meta-Self-Training Based on Teacher–Student Network for Industrial Label-Noise Fault Diagnosis

Pu, Xiaokun; Li, Chunguang

doi:10.1109/tim.2022.3232654

Cited by 6 publications

(3 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some supervised learning models attempt to address the problem of noisy labels. For example, in MST-TS [79], a meta-self-training method is introduced, which employs a self-training mechanism to train a teacher network and leverages the pseudo-labels generated by the teacher to train a student network. In the future, we will improve the robustness of our unsupervised AD model to noisy data.…”

Section: J Limitationsmentioning

confidence: 99%

Anomaly Detection for Medical Images Using Teacher–Student Model With Skip Connections and Multiscale Anomaly Consistency

Liu,

Jiao,

et al. 2024

IEEE Trans. Instrum. Meas.

View full text Add to dashboard Cite

Anomaly detection (AD) in medical images aims to recognize test-time abnormal inputs according to normal samples in the training set. Knowledge distillation based on the teacher-student (T-S) model is a simple and effective method to identify anomalies, yet its efficacy is constrained by the similarity between teacher and student network architectures. To address this problem, in this paper, we propose a T-S model with skip connections (Skip-TS) which is trained by direct reverse knowledge distillation (DRKD) for AD in medical images. First, to overcome the low sensitivity to anomalies caused by structural similarity, we design an encoder-decoder architecture where the teacher network (T-Net) is a pre-trained encoder and the student network (S-Net) is a randomly initialized decoder. During training, the S-Net learns to reconstruct the shallow representations of images from the output of the T-Net, which is called DRKD. Secondly, we introduce skip connections to the T-S model to prevent the S-Net from missing normal information of images at multiscale. In addition, we design a multi-scale anomaly consistency (MAC) loss to improve the anomaly detection and localization performance. Thorough experiments conducted on twelve public medical datasets and two private medical datasets demonstrate that our approach surpasses the current state-of-the-art by 6.4% and 8.2% in terms of AUROC on public and private datasets, respectively. Code and organized benchmark datasets will be available at https://github.com/Arktis2022/Skip-TS.

show abstract

Section: J Limitationsmentioning

confidence: 99%

Anomaly Detection for Medical Images Using Teacher–Student Model With Skip Connections and Multiscale Anomaly Consistency

Liu,

Jiao,

et al. 2024

IEEE Trans. Instrum. Meas.

View full text Add to dashboard Cite

show abstract

“…Some supervised learning models attempt to address the problem of noisy labels. For example, in MST-TS [72], a meta-self-training method is introduced, which employs a self-training mechanism to train a teacher network and leverages the pseudo-labels generated by the teacher to train a student network. In the future, we will improve the robustness of our unsupervised AD model to noisy data.…”

Section: J Limitationsmentioning

confidence: 99%

Anomaly Detection for Medical Images Using Teacher-Student Model with Skip Connections and Multi-scale Anomaly Consistency

Liu,

Jiao,

et al. 2023

Preprint

View full text Add to dashboard Cite

<p>Anomaly detection (AD) in medical images aims to recognize test-time abnormal inputs according to normal samples in the training set. Knowledge distillation based on the teacher-student (T-S) model is a simple and effective method to identify anomalies, yet its efficacy is constrained by the similarity between teacher and student network architectures. To address this problem, in this paper, we propose a T-S model with skip connections (Skip-TS) which is trained by direct reverse knowledge distillation (DRKD) for AD in medical images. First, to overcome the low sensitivity to anomalies caused by structural similarity, we design an encoder-decoder architecture where the teacher network (T-Net) is a pre-trained encoder and the student network (S-Net) is a randomly initialized decoder. During training, the S-Net learns to reconstruct the shallow representations of images from the output of the T-Net, which is called DRKD. Secondly, we introduce skip connections to the T-S model to prevent the S-Net from missing normal information of images at multi-scale. In addition, we design a multi-scale anomaly consistency (MAC) loss to improve the anomaly detection and localization performance. Thorough experiments conducted on twelve public medical datasets and two private medical datasets demonstrate that our approach surpasses the current state-of-the-art by 6.4% and 8.2% in terms of AUROC on public and private datasets, respectively. Code and organized benchmark datasets will be available at https://github.com/Arktis2022/Skip-TS.</p>

show abstract

“…The urgent problems to be solved are how to guide the DNNs to avoid overfitting noisy labels and learn valuable features from noisy dataset. Pu and Li [19] introduced a meta-self-training approach, which combined self-training scheme and pseudolabels to train teacher and student networks, and employed pseudo-labels instead of noisy labels to minimize the negative effects of noisy labels. It addresses not only the recognition bias for self-training with noise-containing scenes, but also obtains better label correction results.…”

Section: Introductionmentioning

confidence: 99%

A robust intelligent fault diagnosis method for rotating machinery under noisy labels

Chen,

Wang,

Ruan

et al. 2023

Meas. Sci. Technol.

View full text Add to dashboard Cite

Despite achieving considerable success, the fault diagnosis methods will still be disturbed by noisy labels, this causes the model’s degradation and reduced diagnostic precision. Focused on solving the above issues, a robust intelligent fault diagnosis approach for rotating machinery under noisy labels is proposed. Firstly, we maintain two deep neural networks (DNNs) and alternatively execute parameters updating and models optimization by referring to the Co-teaching strategy, which can maximize filtering different error types and implement pre-training of DNNs. Secondly, adopting a two-component Gaussian mixture model (GMM) to fit training dataset’s cross-entropy (CE) loss and realize the clean and noisy labels division according to the threshold. Then, a data augmentation method called Mixup operation is employed in semi-supervised learning (SSL) to increase noise robustness and avoid error accumulation, subsequently, performing fine-tuning and correction for clean and noisy samples. Challenging experiments on a transmission gearbox dataset under different noisy labels levels show that the proposed method has robustness to noise and significantly surpasses other approaches, which provides an important reference value for accurate fault diagnosis for rotating equipment parts with noisy labels.

show abstract

Meta-Self-Training Based on Teacher–Student Network for Industrial Label-Noise Fault Diagnosis

Cited by 6 publications

References 25 publications

Anomaly Detection for Medical Images Using Teacher–Student Model With Skip Connections and Multiscale Anomaly Consistency

Anomaly Detection for Medical Images Using Teacher–Student Model With Skip Connections and Multiscale Anomaly Consistency

Anomaly Detection for Medical Images Using Teacher-Student Model with Skip Connections and Multi-scale Anomaly Consistency

A robust intelligent fault diagnosis method for rotating machinery under noisy labels

Contact Info

Product

Resources

About