Global-and-Local Collaborative Learning for Co-Salient Object Detection

Cong, Runmin; Yang, Ning; Li, Chongyi; Fu, Huazhu; Zhao, Yao; Huang, Qingming

doi:10.1109/tcyb.2022.3169431

Cited by 53 publications

(16 citation statements)

References 73 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We also identify other two-modality SOD tasks (e.g., imagevideo optical flow SOD [89], text-image SOD [90], audioimage SOD [91]) and the similar but different image pair co-saliency task [92]- [94]. Therefore, two-modality SOD is worthy of further study.…”

Section: B Two-modality Salient Object Detectionmentioning

confidence: 99%

HRTransNet: HRFormer-Driven Two-Modality Salient Object Detection

Tang

Liu

Tan

et al. 2023

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

High-Resolution Transformer (HRFormer) can maintain high-resolution representation and share global receptive fields. It is friendly towards salient object detection (SOD) in which the input and output have the same resolution. However, two critical problems need to be solved for two-modality SOD. One problem is two-modality fusion. The other problem is the HRFormer output's fusion. To address the first problem, a supplementary modality is injected into the primary modality by using global optimization and an attention mechanism to select and purify the modality at the input level. To solve the second problem, a dual-direction short connection fusion module is used to optimize the output features of HRFormer, thereby enhancing the detailed representation of objects at the output level. The proposed model, named HRTransNet, first introduces an auxiliary stream for feature extraction of supplementary modality. Then, features are injected into the primary modality at the beginning of each multi-resolution branch. Next, HRFormer is applied to achieve forwarding propagation. Finally, all the output features with different resolutions are aggregated by intrafeature and inter-feature interactive transformers. Application of the proposed model results in impressive improvement for driving two-modality SOD tasks, e.g., RGB-D, RGB-T, and light field SOD.https://github.com/liuzywen/HRTransNet

show abstract

Section: B Two-modality Salient Object Detectionmentioning

confidence: 99%

HRTransNet: HRFormer-Driven Two-Modality Salient Object Detection

Tang

Liu

Tan

et al. 2023

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

show abstract

“…where G is the ground truth, and bce is the BCE loss as defined in [22], [24]. During the testing phase, we only utilize the prediction of RGB-D stream as the final saliency map.…”

Section: E Loss Functionmentioning

confidence: 99%

“…With the development of SOD task research, there are many subtasks, such as co-salient object detection (CoSOD) [20]- [22], remote sensing SOD [23]- [27], video SOD [28]- [30], light field SOD [31], have also been developed. In fact, the natural binocular structure of humans can also perceive the depth of field of the scene, and then generate stereo perception.…”

Section: Introductionmentioning

confidence: 99%

CIR-Net: Cross-modality Interaction and Refinement for RGB-D Salient Object Detection

Cong¹,

Lin²,

Zhang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Focusing on the issue of how to effectively capture and utilize cross-modality information in RGB-D salient object detection (SOD) task, we present a convolutional neural network (CNN) model, named CIR-Net, based on the novel cross-modality interaction and refinement. For the cross-modality interaction, 1) a progressive attention guided integration unit is proposed to sufficiently integrate RGB-D feature representations in the encoder stage, and 2) a convergence aggregation structure is proposed, which flows the RGB and depth decoding features into the corresponding RGB-D decoding streams via an importance gated fusion unit in the decoder stage. For the cross-modality refinement, we insert a refinement middleware structure between the encoder and the decoder, in which the RGB, depth, and RGB-D encoder features are further refined by successively using a self-modality attention refinement unit and a cross-modality weighting refinement unit. At last, with the gradually refined features, we predict the saliency map in the decoder stage. Extensive experiments on six popular RGB-D SOD benchmarks demonstrate that our network outperforms the state-of-the-art saliency detectors both qualitatively and quantitatively. The code and results can be found from the link of https://rmcong.github. io/proj CIRNet.html.

show abstract

“…With the success of unified models in up-stream tasks (Ren et al 2015;Xiao et al 2017), the latest CoSOD models try to address salient object detection and common object detection in a unified framework (Fan et al 2021(Fan et al , 2022Zhang et al 2020c). Despite the promising performance achieved by these methods, most of them only focus on learning better consistent feature representations in an individual group (Zhang et al 2020c;Wei et al 2017;Zhang et al 2020b;Cong et al 2022;Jin et al 2020;Tang et al 2022), which may make them suffer from the following limitations. First, images from the same group can only act as positive samples of each other.…”

Section: Introductionmentioning

confidence: 99%

Memory-aided Contrastive Consensus Learning for Co-salient Object Detection

Zheng¹,

Qin²,

Wang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Co-Salient Object Detection (CoSOD) aims at detecting common salient objects within a group of relevant source images. Most of the latest works employ the attention mechanism for finding common objects. To achieve accurate CoSOD results with high-quality maps and high efficiency, we propose a novel Memory-aided Contrastive Consensus Learning (MCCL) framework, which is capable of effectively detecting co-salient objects in real time (∼110 fps). To learn better group consensus, we propose the Group Consensus Aggregation Module (GCAM) to abstract the common features of each image group; meanwhile, to make the consensus representation more discriminative, we introduce the Memory-based Contrastive Module (MCM), which saves and updates the consensus of images from different groups in a queue of memories. Finally, to improve the quality and integrity of the predicted maps, we develop an Adversarial Integrity Learning (AIL) strategy to make the segmented regions more likely composed of complete objects with less surrounding noise. Extensive experiments on all the latest CoSOD benchmarks demonstrate that our lite MCCL outperforms 13 cutting-edge models, achieving the new state of the art (∼5.9% and ∼6.2% improvement in S-measure on CoSOD3k and CoSal2015, respectively). Our source codes, saliency maps, and online demos are publicly available at https://github.com/ZhengPeng7/MCCL.

show abstract

Global-and-Local Collaborative Learning for Co-Salient Object Detection

Cited by 53 publications

References 73 publications

HRTransNet: HRFormer-Driven Two-Modality Salient Object Detection

HRTransNet: HRFormer-Driven Two-Modality Salient Object Detection

CIR-Net: Cross-modality Interaction and Refinement for RGB-D Salient Object Detection

Memory-aided Contrastive Consensus Learning for Co-salient Object Detection

Contact Info

Product

Resources

About