OSFormer: One-Stage Camouflaged Instance Segmentation with Transformers

Pei, Jialun; Cheng, Tianyang; Fan, Deng-Ping; Tang, He; Chen, Chuanbo; Gool, Luc Van

doi:10.1007/978-3-031-19797-0_2

Cited by 30 publications

(7 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As shown in Table 2, our IA-guided queries achieve better results than zero [4] or learning-based ones [9]. Recent work [33] proposes to use resized multiscale features as instance queries. However, such a fixposition query selection strategy is hard to extract representative embeddings for all potential objects and thus obtains lower performance.…”

Section: Ablation Studiesmentioning

confidence: 82%

FastInst: A Simple Query-Based Model for Real-Time Instance Segmentation

He¹,

Li²,

Geng³

et al. 2023

Preprint

View full text Add to dashboard Cite

Recent attention in instance segmentation has focused on query-based models. Despite being non-maximum suppression (NMS)-free and end-to-end, the superiority of these models on high-accuracy real-time benchmarks has not been well demonstrated. In this paper, we show the strong potential of query-based models on efficient instance segmentation algorithm designs. We present FastInst, a simple, effective query-based framework for real-time instance segmentation. FastInst can execute at a real-time speed (i.e., 32.5 FPS) while yielding an AP of more than 40 (i.e., 40.5 AP) on COCO test-dev without bells and whistles. Specifically, FastInst follows the meta-architecture of recently introduced Mask2Former. Its key designs include instance activation-guided queries, dual-path update strategy, and ground truth mask-guided learning, which enable us to use lighter pixel decoders, fewer Transformer decoder layers, while achieving better performance. The experiments show that FastInst outperforms most state-ofthe-art real-time counterparts, including strong fully convolutional baselines, in both speed and accuracy. Code can be found at https://github.com/junjiehe96/ FastInst.

show abstract

Section: Ablation Studiesmentioning

confidence: 82%

FastInst: A Simple Query-Based Model for Real-Time Instance Segmentation

He¹,

Li²,

Geng³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Sun et al [45] placed emphasis on rich global context information with the integration of cross-level features. Pei et al [38] applied a one-stage location-sensing transformer and further fused the features from transformer and CNN. Some bio-inspired methods are proposed.…”

Section: Related Workmentioning

confidence: 99%

“…Recently, many researches put emphasis on learning from a fixed single view with either auxiliary tasks [18,32,34,58,67,15], uncertainty discovery [20,26], or vision transformers [56,38] and their proposed methods achieved significant progress. Nevertheless, due to visual insignificance of camouflaged objects and contextual insufficiency from single-view input, they are still striving to precisely recognize camouflaged objects and their performance needs to be improved.…”

Section: Introductionmentioning

confidence: 99%

MFFN: Multi-view Feature Fusion Network for Camouflaged Object Detection

Zheng¹,

Zheng²,

Yang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent research about camouflaged object detection (COD) aims to segment highly concealed objects hidden in complex surroundings. The tiny, fuzzy camouflaged objects result in visually indistinguishable properties. However, current single-view COD detectors are sensitive to background distractors. Therefore, blurred boundaries and variable shapes of the camouflaged objects are challenging to be fully captured with a singleview detector. To overcome these obstacles, we propose a behavior-inspired framework, called Multi-view Feature Fusion Network (MFFN), which mimics the human behaviors of finding indistinct objects in images, i.e., observing from multiple angles, distances, perspectives. Specifically, the key idea behind it is to generate multiple ways of observation (multi-view) by data augmentation and apply them as inputs. MFFN captures critical boundary and semantic information by comparing and fusing extracted multi-view features. In addition, our MFFN exploits the dependence and interaction between views and channels. Specifically, our methods leverage the complementary information between different views through a two-stage attention module called Co-attention of Multi-view (CAMV). And we design a local-overall module called Channel Fusion Unit (CFU) to explore the channel-wise contextual clues of diverse feature maps in an iterative manner. The experiment results show that our method performs favorably against existing state-of-the-art methods via training with the same data. The code will be available at https://github.com/ dwardzheng/MFFN_COD.

show abstract

“…Then a series of improved versions sprung up, such as data-efficient image transformers (DeiT) [40], pyramid vision transformer [42], and Swin transformer [23]. For camouflaged object detection, [32] propose a one-stage transformer framework for camouflaged instance segmentation. [47], [24], and [14] have made some attempts to detect camouflaged objects using transformers and achieved good performance.…”

Section: Vision Transformermentioning

confidence: 99%

Horizontal Feature Pyramid Network for Object Detection in UAV Images

Tang

Sun

Wang

2021

2021 China Automation Congress (CAC)

View full text Add to dashboard Cite

Vision transformers have recently shown strong global context modeling capabilities in camouflaged object detection. However, they suffer from two major limitations: less effective locality modeling and insufficient feature aggregation in decoders, which are not conducive to camouflaged object detection that explores subtle cues from indistinguishable backgrounds. To address these issues, in this paper, we propose a novel transformer-based Feature Shrinkage Pyramid Network (FSPNet), which aims to hierarchically decode locality-enhanced neighboring transformer features through progressive shrinking for camouflaged object detection. Specifically, we propose a nonlocal token enhancement module (NL-TEM) that employs the non-local mechanism to interact neighboring tokens and explore graph-based high-order relations within tokens to enhance local representations of transformers. Moreover, we design a feature shrinkage decoder (FSD) with adjacent interaction modules (AIM), which progressively aggregates adjacent transformer features through a layer-bylayer shrinkage pyramid to accumulate imperceptible but effective cues as much as possible for object information decoding. Extensive quantitative and qualitative experiments demonstrate that the proposed model significantly outperforms the existing 24 competitors on three challenging COD benchmark datasets under six widely-used evaluation metrics. Our code is publicly available at https: //github.com/ZhouHuang23/FSPNet.

show abstract

OSFormer: One-Stage Camouflaged Instance Segmentation with Transformers

Cited by 30 publications

References 53 publications

FastInst: A Simple Query-Based Model for Real-Time Instance Segmentation

FastInst: A Simple Query-Based Model for Real-Time Instance Segmentation

MFFN: Multi-view Feature Fusion Network for Camouflaged Object Detection

Horizontal Feature Pyramid Network for Object Detection in UAV Images

Contact Info

Product

Resources

About