Fast target-aware learning for few-shot video object segmentation

Abstract

Few-shot video object segmentation (FSVOS) aims to segment a specific object throughout a video sequence when only the first-frame annotation is given. In this study, we develop a fast target-aware learning approach for FSVOS, where the proposed approach adapts to new video sequences from its firstframe annotation through a lightweight procedure. The proposed network comprises two models. First, the meta knowledge model learns the general semantic features for the input video image and up-samples the coarse predicted mask to the original image size. Second, the target model adapts quickly from the limited support set. Concretely, during the online inference for testing the video, we first employ fast optimization techniques to train a powerful target model by minimizing the segmentation error in the first frame and then use it to predict the subsequent frames. During the offline training, we use a bilevel-optimization strategy to mimic the full testing procedure to train the meta knowledge model across multiple video sequences. The proposed method is trained only on an individual public video object segmentation (VOS) benchmark without additional training sets and compared favorably with state-of-the-art methods on DAVIS-2017, with a J &F overall score of 71.6%, and on YouTubeVOS-2018, with a J &F overall score of 75.4%. Meanwhile, a high inference speed of approximately 0.13 s per frame is maintained.

Mitigating Distractor Challenges in Video Object Segmentation through Shape and Motion Cues

“…Recent research in matching-based video object segmentation (VOS) [1][2][3][4][5][6][7][8][9][17][18][19][20][21][22][23] has primarily focused on improving network structures, including building more efficient memory [5,6], adopting local matching [7][8][9], and incorporating background context [3,4]. Among them, STM [1] is the pioneer in the field of video object segmentation, which first proposed the design of the memory bank, enabling VOS methods to effectively utilize the temporal information in video sequences.…”

Spatial-Aware Transformer (SAT): Enhancing Global Modeling in Transformer Segmentation for Remote Sensing Images

“…Convolutional Neural Networks (CNNs) [11,12,18,19] have long been recognized as powerful tools for image analysis due to their ability to learn hierarchical feature representations from raw image data. They have been particularly successful in various segmentation tasks, thanks to their robustness in extracting spatial features from images.…”

“…The self-attention mechanism computes the response at a position as a weighted sum of the features at all positions in the data. This global context-awareness allows the Transformer to better capture intricate spatial structures and long-range dependencies that are characteristic of remote sensing images [19,[29][30][31].…”