Don't fear the unlabelled: safe deep semi-supervised learning via simple debiasing

Abstract

Semi supervised learning (SSL) provides an effective means of leveraging unlabelled data to improve a model's performance. Even though the domain has received a considerable amount of attention in the past years, most methods present the common drawback of being unsafe. By safeness we mean the quality of not degrading a fully supervised model when including unlabelled data. Our starting point is to notice that the estimate of the risk that most discriminative SSL methods minimise is biased, even asymptotically. This bias makes these techniques untrustable without a proper validation set, but we propose a simple way of removing the bias. Our debiasing approach is straightforward to implement, and applicable to most deep SSL methods. We provide simple theoretical guarantees on the safeness of these modified methods, without having to rely on the strong assumptions on the data distribution that SSL theory usually requires. We evaluate debiased versions of different existing SSL methods and show that debiasing can compete with classic deep SSL techniques in various classic settings and even performs well when traditional SSL fails.

MVCRNet: A Semi-Supervised Multi-View Framework for Robust Animal Pose Estimation with Minimal Labeled Data

“…Bold indicates the best performance. 0.458 0.490 0.551 0.585 0.602 0.634 0.617 0.647 0.644 0.672 UDA [16] 0.513 0.555 0.573 0.613 0.612 0.646 0.637 0.669 0.652 0.682 FixMatch [17] 0.509 0.551 0.574 0.610 0.610 0.644 0.641 0.673 0.653 0.683 DualPose [25] 0.495 0.539 0.573 0.610 0.613 0.648 0.630 0.664 0.652 0.682 DefixMatch [31] 0.456 0.496 0.557 0.597 0.579 0.612 0.623 0.652 0.647 0.678 FreeMatch [18] 0.497 0.537 0.576 0.612 0.608 0.642 0.625 0.655 0.653 0.682 ScarceNet [26] 0.561 0.604 0.596 0.640 0.635 0.674 0.657 0.697 0.671 0.708 Ours 0.574 0.608 0.638 0.672 0.660 0.693 0.676 0.706 0.690 0.720 Fig. 3: Qualitative results obtained after training our method on the AP10K dataset using only 25 labeled images per species.…”