Dual Student: Breaking the Limits of the Teacher in Semi-Supervised Learning

Ke, Zhanghan; Wang, Daoye; Yan, Qing; Ren, Jimmy; Lau, Rynson W. H.

doi:10.1109/iccv.2019.00683

Cited by 194 publications

(138 citation statements)

References 17 publications

Supporting

Mentioning

129

Contrasting

Unclassified

Order By: Relevance

“…[35] developed Mean Teacher that takes an exponential moving average of model weights to obtain more accurate predictions for consistency regularization. Different from Mean Teacher, [15] decoupled the connection between student and teacher networks, and used another student network to substitute for the teacher model. [36] proposed Interpolation Consistency Training which enforces the prediction at an interpolation of unlabeled points to match with the interpolation of the predictions at those points.…”

Section: Ssl In Natural Image Classificationmentioning

confidence: 99%

SSMD: Semi-Supervised Medical Image Detection with Adaptive Consistency and Heterogeneous Perturbation

Wang¹,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Semi-Supervised classification and segmentation methods have been widely investigated in medical image analysis. Both approaches can improve the performance of fully-supervised methods with additional unlabeled data. However, as a fundamental task, semi-supervised object detection has not gained enough attention in the field of medical image analysis. In this paper, we propose a novel Semi-Supervised Medical image Detector (SSMD). The motivation behind SSMD is to provide free yet effective supervision for unlabeled data, by regularizing the predictions at each position to be consistent. To achieve the above idea, we develop a novel adaptive consistency cost function to regularize different components in the predictions. Moreover, we introduce heterogeneous perturbation strategies that work in both feature space and image space, so that the proposed detector is promising to produce powerful image representations and robust predictions. Extensive experimental results show that the proposed SSMD achieves the state-of-the-art performance at a wide range of settings. We also demonstrate the strength of each proposed module with comprehensive ablation studies.

show abstract

Section: Ssl In Natural Image Classificationmentioning

confidence: 99%

SSMD: Semi-Supervised Medical Image Detection with Adaptive Consistency and Heterogeneous Perturbation

Wang¹,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…However, the teacher model is usually fixed, making the student inherit some inaccurate predictions from the teacher. Recent efforts turn to update the teacher along with the student in order to generate better pseudo labels, e.g., co-teaching [48,49], dual student [50], meta pseudo label [10], mutual training [18,19] and so on. Such mutual learning strategy however may end up learning homogeneous knowledge and hence fail to provide complementary information for each other.…”

Section: Related Workmentioning

confidence: 99%

Robust Adversarial Learning For Semi-Supervised Semantic Segmentation

Zhang

et al. 2020

2020 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Recent semi-supervised learning (SSL) methods are commonly based on pseudo labeling. Since the SSL performance is greatly influenced by the quality of pseudo labels, mutual learning has been proposed to effectively suppress the noises in the pseudo supervision. In this work, we propose robust mutual learning that improves the prior approach in two aspects. First, the vanilla mutual learners suffer from the coupling issue that models may converge to learn homogeneous knowledge. We resolve this issue by introducing mean teachers to generate mutual supervisions so that there is no direct interaction between the two students. We also show that strong data augmentations, model noises and heterogeneous network architectures are essential to alleviate the model coupling. Second, we notice that mutual learning fails to leverage the network's own ability for pseudo label refinement. Therefore, we introduce self-rectification that leverages the internal knowledge and explicitly rectifies the pseudo labels before the mutual teaching. Such self-rectification and mutual teaching collaboratively improve the pseudo label accuracy throughout the learning. The proposed robust mutual learning demonstrates state-of-the-art performance on semantic segmentation in low-data regime.

show abstract

“…In the implementation of e-learning learning, especially early childhood teachers need to implement a brain-turning system because learning that was originally implemented in the form of offline was changed to online learning or e-learning. In its implementation the teacher also needs to make a simple implementation plan of learning both daily and weekly given to parents so that parents can also carry out what is expected by the teacher to the child (Ke et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

Early Childhood Education Teachers Consistency of E-Learning Programs

Widiastuti

Rasmani

Wahyuningsih

2021

View full text Add to dashboard Cite

The e-learning program is a new program implemented at the early childhood education level. The teacher of the main character in the application of e-learning has a dilemma in running this program. The focus of the study identified the consistency of early childhood education teachers in implementing e-learning programs during the Covid-19 pandemic. Using quantitative methods with purposive sampling technique. Data obtained through interviews (G-Form) with respondents 50 ECE teachers in the city of Surakarta. Field facts are that there is no teacher consistency in implementing e-learning programs such as the absence of daily or weekly learning plans in implementing e-learning, uncertain teaching schedules, and unsupported learning media. This shows that 80% of ECE teachers implement e-learning and 20% implement mixed learning. Teacher consistency can be improved through training provided by the government, provision of educational facilities by schools, learning innovations according to health protocols and initial motivation to teach

show abstract

Dual Student: Breaking the Limits of the Teacher in Semi-Supervised Learning

Cited by 194 publications

References 17 publications

SSMD: Semi-Supervised Medical Image Detection with Adaptive Consistency and Heterogeneous Perturbation

SSMD: Semi-Supervised Medical Image Detection with Adaptive Consistency and Heterogeneous Perturbation

Robust Adversarial Learning For Semi-Supervised Semantic Segmentation

Early Childhood Education Teachers Consistency of E-Learning Programs

Contact Info

Product

Resources

About