Ada-Segment: Automated Multi-loss Adaptation for Panoptic Segmentation

Abstract

Panoptic segmentation that unifies instance segmentation and semantic segmentation has recently attracted increasing attention. While most existing methods focus on designing novel architectures, we steer toward a different perspective: performing automated multi-loss adaptation (named Ada-Segment) on the fly to flexibly adjust multiple training losses over the course of training using a controller trained to capture the learning dynamics. This offers a few advantages: it bypasses manual tuning of the sensitive loss combination, a decisive factor for panoptic segmentation; it allows to explicitly model the learning dynamics, and reconcile the learning of multiple objectives (up to ten in our experiments); with an end-to-end architecture, it generalizes to different datasets without the need of re-tuning hyperparameters or readjusting the training process laboriously. Our Ada-Segment brings 2.7% panoptic quality (PQ) improvement on COCO val split from the vanilla baseline, achieving the state-of-theart 48.5% PQ on COCO test-dev split and 32.9% PQ on ADE20K dataset. The extensive ablation studies reveal the ever-changing dynamics throughout the training process, necessitating the incorporation of an automated and adaptive learning strategy as presented in this paper.

MFNet: Panoptic segmentation network based on multiscale feature weighted fusion and frequency domain attention mechanism

“…It can be seen from the data in Table 2 that the overall performance of MFNet is better than that of most panoptic segmentation networks. At the same time, compared with SOGNet [10] and Ada-Segment [15], which show the best performance, the panoptic segmentation quality of MFNet is only 0.3% lower. In terms of the panoptic quality corresponding to the stuff class, MFNet is better than all existing networks.…”

“…OCFusion [14] establishes a binary relationship between instance masks to determine the occlusion order between them and then solves the problem of instance overlap in panoptic segmentation. Ada-Segment [15] and AdaptIs [16] enhance the segmentation performance of panoptic segmentation by introducing adaptive algorithms.…”

HLIC: Harmonizing Optimization Metrics in Learned Image Compression by Reinforcement Learning

“…Wang et al [38] further improves this method for face recognition by redesigning the search space based on deeper analysis of margin-based softmax losses. Very recently, Ada-Segment [41] extends the idea of AM-LFS to perform multi-loss adaptation for panoptic segmentation. Auto Seg-Loss [21] proposes to search for parameterized metric-specific surrogate loss in semantic segmentation.…”