Self-Training With Noisy Student Improves ImageNet Classification

Xie, Qizhe; Luong, Minh-Thang; Hovy, Eduard; Le, Quoc V.

doi:10.1109/cvpr42600.2020.01070

Cited by 1,882 publications

(1,488 citation statements)

References 30 publications

Supporting

Mentioning

1,282

Contrasting

Unclassified

Order By: Relevance

“…Recent studies adopt a teacher-student training paradigm, i.e., pseudolabels are generated by the teacher model on the unlabeled dataset, which is then combined with the labeled data and used to train or finetune the student model. For example, an iterative training scheme is proposed in [35], where the trained student model is used as the teacher model at the subsequent training round. The method outperforms the fully supervised counterpart on ImageNet by a large margin.…”

Section: B Learningmentioning

confidence: 99%

Empowering Things With Intelligence: A Survey of the Progress, Challenges, and Opportunities in Artificial Intelligence of Things

Zhang

Tao

2021

IEEE Internet Things J.

419

168

View full text Add to dashboard Cite

In the Internet of Things (IoT) era, billions of sensors and devices collect and process data from the environment, transmit them to cloud centers, and receive feedback via the internet for connectivity and perception. However, transmitting massive amounts of heterogeneous data, perceiving complex environments from these data, and then making smart decisions in a timely manner are difficult. Artificial intelligence (AI), especially deep learning, is now a proven success in various areas including computer vision, speech recognition, and natural language processing. AI introduced into the IoT heralds the era of artificial intelligence of things (AIoT). This paper presents a comprehensive survey on AIoT to show how AI can empower the IoT to make it faster, smarter, greener, and safer. Specifically, we briefly present the AIoT architecture in the context of cloud computing, fog computing, and edge computing. Then, we present progress in AI research for IoT from four perspectives: perceiving, learning, reasoning, and behaving. Next, we summarize some promising applications of AIoT that are likely to profoundly reshape our world. Finally, we highlight the challenges facing AIoT and some potential research opportunities.

show abstract

Section: B Learningmentioning

confidence: 99%

Empowering Things With Intelligence: A Survey of the Progress, Challenges, and Opportunities in Artificial Intelligence of Things

Zhang

Tao

2021

IEEE Internet Things J.

419

168

View full text Add to dashboard Cite

show abstract

“…Offline training. Image classifiers used by companies, such Facebook or Google, are trained on tens of million of labeled images (Kuznetsova et al, 2020), or pretrained on billions of images (Mahajan et al, 2018;Xie et al, 2020), to reach the level of quality required by certain products. Natural language processing (NLP) systems for machine translation, or speech recognition systems such as BERT (Devlin et al, 2019), also require billions of samples to generalize and have descent performance for real applications.…”

mentioning

confidence: 99%

How to train your robot with deep reinforcement learning: lessons we have learned

Ibarz

Tan

Finn

et al. 2021

The International Journal of Robotics Research

349

131

View full text Add to dashboard Cite

Deep reinforcement learning (RL) has emerged as a promising approach for autonomously acquiring complex behaviors from low-level sensor observations. Although a large portion of deep RL research has focused on applications in video games and simulated control, which does not connect with the constraints of learning in real environments, deep RL has also demonstrated promise in enabling physical robots to learn complex skills in the real world. At the same time, real-world robotics provides an appealing domain for evaluating such algorithms, as it connects directly to how humans learn: as an embodied agent in the real world. Learning to perceive and move in the real world presents numerous challenges, some of which are easier to address than others, and some of which are often not considered in RL research that focuses only on simulated domains. In this review article, we present a number of case studies involving robotic deep RL. Building off of these case studies, we discuss commonly perceived challenges in deep RL and how they have been addressed in these works. We also provide an overview of other outstanding challenges, many of which are unique to the real-world robotics setting and are not often the focus of mainstream RL research. Our goal is to provide a resource both for roboticists and machine learning researchers who are interested in furthering the progress of deep RL in the real world.

show abstract

“…Most SSLs take a pseudo label learning approach [6,19,16,30]. For example, pseudo labeling [19] (also called self-training [36]) first trains using labeled data and then confident unlabeled data, which is larger than a predefined threshold, is added to the training data and the classifier is trained iteratively. Consistency regularization [21] generates pseudo labeled samples on the basis of the idea that a classifier should output the same class distribution for an unlabeled example even after it has been augmented.…”

Section: Related Workmentioning

confidence: 99%

Negative Pseudo Labeling Using Class Proportion for Semantic Segmentation in Pathology

Tokunaga

Iwana

Teramoto

et al. 2020

Computer Vision – ECCV 2020

View full text Add to dashboard Cite

In pathological diagnosis, since the proportion of the adenocarcinoma subtypes is related to the recurrence rate and the survival time after surgery, the proportion of cancer subtypes for pathological images has been recorded as diagnostic information in some hospitals. In this paper, we propose a subtype segmentation method that uses such proportional labels as weakly supervised labels. If the estimated class rate is higher than that of the annotated class rate, we generate negative pseudo labels, which indicate, "input image does not belong to this negative label," in addition to standard pseudo labels. It can force out the low confidence samples and mitigate the problem of positive pseudo label learning which cannot label low confident unlabeled samples. Our method outperformed the state-of-the-art semi-supervised learning (SSL) methods.

show abstract

Self-Training With Noisy Student Improves ImageNet Classification

Cited by 1,882 publications

References 30 publications

Empowering Things With Intelligence: A Survey of the Progress, Challenges, and Opportunities in Artificial Intelligence of Things

Empowering Things With Intelligence: A Survey of the Progress, Challenges, and Opportunities in Artificial Intelligence of Things

How to train your robot with deep reinforcement learning: lessons we have learned

Negative Pseudo Labeling Using Class Proportion for Semantic Segmentation in Pathology

Contact Info

Product

Resources

About