MUM : Mix Image Tiles and UnMix Feature Tiles for Semi-Supervised Object Detection

Abstract

Many recent semi-supervised learning (SSL) studies build teacher-student architecture and train the student network by the generated supervisory signal from the teacher. Data augmentation strategy plays a significant role in the SSL framework since it is hard to create a weak-strong augmented input pair without losing label information. Especially when extending SSL to semi-supervised object detection (SSOD), many strong augmentation methodologies related to image geometry and interpolation-regularization are hard to utilize since they possibly hurt the location information of the bounding box in the object detection task. To address this, we introduce a simple yet effective data augmentation method, Mix/UnMix (MUM), which unmixes feature tiles for the mixed image tiles for the SSOD framework. Our proposed method makes mixed input image tiles and reconstructs them in the feature space. Thus, MUM can enjoy the interpolation-regularization effect from non-interpolated pseudo-labels and successfully generate a meaningful weak-strong pair. Furthermore, MUM can be easily equipped on top of various SSOD methods. Extensive experiments on MS-COCO and PASCAL VOC datasets demonstrate the superiority of MUM by consistently improving the mAP performance over the baseline in all the tested SSOD benchmark protocols.

Pose-MUM : Reinforcing Key Points Relationship for Semi-Supervised Human Pose Estimation

“…To mitigate the difficulties, the studies on semisupervised learning (SSL) provide productive insights by making use of unlabeled data along with the labeled training data. To train a student network using a small amount of labeled data, knowledge distillation methods have been developed for effective training of the student network with the help of an experienced teacher network, along with various augmentation methods [1,5,9,10,[13][14][15]17], as will be briefly mentioned in Sec. 2.…”

“…The JC Augment adopts dual model for teacher-student network to decouple the teacher and the student, and Joint Cutout as strong augmentation for simulating hard occlusion of key points by performing Cutout [4,8] at human key points. However, as clarified in [10], Cutout [4] augmentation leads to information loss, and the dual network training method used to overcome the instability of EMA requires double training cost.…”

“…In this paper, to mitigate the difficulties aforementioned, we propose a new augmentation method to reinforce key points relationship for SSHPE in the teacher-student framework, along with building an improved teacher. We adopt MUM [10] to the SSHPE task and develop an advanced version, called Pose-MUM.…”

“…MUM augmentation shows a powerful strategy for strong-weak augmentation in dense prediction tasks of SSL [10]. However, MUM is performed at the image-level, thus may blind connections between any two key points in the whole backbone forward path.…”

“…

A well-designed strong-weak augmentation strategy and the stable teacher to generate reliable pseudo labels are essential in the teacher-student framework of semi-supervised learning (SSL). Considering these in mind, to suit the semisupervised human pose estimation (SSHPE) task, we propose a novel approach referred to as Pose-MUM that modifies Mix/UnMix (MUM) augmentation [10]. Like MUM in the dense prediction task, the proposed Pose-MUM makes strong-weak augmentation for pose estimation and leads the network to learn the relationship between each human key points much better than the conventional methods by adding the mixing process in intermediate layers in a stochastic manner.

…”