Hierarchical Alternate Interaction Network for RGB-D Salient Object Detection

Li, Gongyang; Li, Zhi; Chen, Minyu; Bai, Zhen; Wang, Lin; Ling, Haibin

doi:10.1109/tip.2021.3062689

Cited by 203 publications

(70 citation statements)

References 91 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the NJUD [49] dataset, our meth-od is similar to or worse than the best performance. Through the percentage presentation, our method improves the MaxF index by 2% compared with HAINet [61] on the SIP [22] data set. In Tab.…”

Section: ) Comparison With Advanced Methodsmentioning

confidence: 99%

“…PCF [54], AFNet [55], CPFP [20], MMCI [33], TANet [44], D3Net [22], JL-DCF [13], DMRA [34], S2MA [14], PGAR [56], ICNet [10], DASNet [57], UC-Net [58], DCF [59], DSA2F [60], HAINet [61]. Some of the above methods are trained with subsets of NJU2K [43] and NLPR [44], while others are trained with subsets of NJU2K [48], NLPR [26] and DUTLF-depth [34].…”

Section: Comparison With State-of-art Methodsmentioning

confidence: 99%

See 1 more Smart Citation

LIANet: Layer Interactive Attention Network for RGB-D Salient Object Detection

Han

Wang

et al. 2022

IEEE Access

View full text Add to dashboard Cite

RGB-D salient object detection (SOD) usually describes two modes' classification or regression problem, namely RGB and depth. The existing RGB-D SOD methods use depth hints to increase the detection performance, meanwhile they focus on the quality of little depth maps. In practical application, the interference of various problems in the acquisition process affects the depth map quality, which dramatically reduces the detection effect. In this paper, to minimize interference in depth mapping and emphasize prominent objects in RGB images, we put forward a layered interactive attention network (LIANet). In general, this network consists of three essential parts: feature coding, layered fusion mechanism, and feature decoding. In the feature coding stage, three-dimensional weight is introduced to the features of each layer without adding network parameters, and it is also a lightweight module. The layered fusion mechanism is the most critical part of this paper. RGB and depth maps are used alternately for layered interaction and fusion to enhance RGB feature information and gradually integrate global context information at a single scale. In addition, we also used mixed losses to optimize further and train our model. Finally, a mass of experiments on six standard datasets demonstrated the importance of the method, and a timely detection speed reaches 30 fps on every dataset.

show abstract

Section: ) Comparison With Advanced Methodsmentioning

confidence: 99%

Section: Comparison With State-of-art Methodsmentioning

confidence: 99%

LIANet: Layer Interactive Attention Network for RGB-D Salient Object Detection

Han

Wang

et al. 2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Our model is compared with 16 state-of-the-art RGB-D SOD models, including D3Net [22], ICNet [41], DCMF [6], DRLF [67], SSF [81], SSMA [43], A2dele [57], UCNet [80], CoNet [33], DANet [90], JLDCF [24], EBFSP [31],CDNet [35], HAINet [40], RD3D [10] and DSA2F [61]. To ensure the fairness of the comparison results, the saliency maps of the evaluation are provided by the authors or generated by running source codes.…”

Section: Comparisons With the State-of-the-artmentioning

confidence: 99%

TriTransNet: RGB-D Salient Object Detection with a Triplet Transformer Embedding Network

Liu,

Wang,

et al. 2021

Preprint

View full text Add to dashboard Cite

Salient object detection is the pixel-level dense prediction task which can highlight the prominent object in the scene. Recently U-Net framework is widely used, and continuous convolution and pooling operations generate multi-level features which are complementary with each other. In view of the more contribution of high-level features for the performance, we propose a triplet transformer embedding module to enhance them by learning long-range dependencies across layers. It is the first to use three transformer encoders with shared weights to enhance multi-level features. By further designing scale adjustment module to process the input, devising three-stream decoder to process the output and attaching depth features to color features for the multi-modal fusion, the proposed triplet transformer embedding network (TriTransNet) achieves the state-of-the-art performance in RGB-D salient object detection, and pushes the performance to a new level. Experimental results demonstrate the effectiveness of the proposed modules and the competition of TriTransNet. 1 CCS CONCEPTS• Computing methodologies → Interest point and salient region detections.

show abstract

“…However, directly fusing the depth cues and RGB information would lead to insufficient cross-modality understanding. Several works [17], [24], [37], [48] propose to fuse these two modality encoders with a stage-wise or hierarchical manner. For example, Chen et al [24] propose a progressive fusion strategy in a coarse-to-fine manner for sufficient information learning.…”

Section: Related Workmentioning

confidence: 99%

“…Zhao et al [17] propose a fluid pyramid integration strategy to make full use of depth enhanced features. Li et al [48] tends to utilize the alternate interaction of different network stages for learning relations of different modalities. Besides these fusion strategies, Chen [34] propose to find a disentangled feature representation of each modality and learns to interact the same type of disentangled feature with others.…”

Section: Related Workmentioning

confidence: 99%

RGB-D Salient Object Detection with Ubiquitous Target Awareness

Zhao,

et al. 2021

Preprint

View full text Add to dashboard Cite

Conventional RGB-D salient object detection methods aim to leverage depth as complementary information to find the salient regions in both modalities. However, the salient object detection results heavily rely on the quality of captured depth data which sometimes are unavailable. In this work, we make the first attempt to solve the RGB-D salient object detection problem with a novel depth-awareness framework. This framework only relies on RGB data in the testing phase, utilizing captured depth data as supervision for representation learning. To construct our framework as well as achieving accurate salient detection results, we propose a Ubiquitous Target Awareness (UTA) network to solve three important challenges in RGB-D SOD task: 1) a depth awareness module to excavate depth information and to mine ambiguous regions via adaptive depth-error weights, 2) a spatial-aware cross-modal interaction and a channel-aware cross-level interaction, exploiting the low-level boundary cues and amplifying high-level salient channels, and 3) a gated multiscale predictor module to perceive the object saliency in different contextual scales. Besides its high performance, our proposed UTA network is depth-free for inference and runs in realtime with 43 FPS. Experimental evidence demonstrates that our proposed network not only surpasses the state-of-the-art methods on five public RGB-D SOD benchmarks by a large margin, but also verifies its extensibility on five public RGB SOD benchmarks.

show abstract

Hierarchical Alternate Interaction Network for RGB-D Salient Object Detection

Cited by 203 publications

References 91 publications

LIANet: Layer Interactive Attention Network for RGB-D Salient Object Detection

LIANet: Layer Interactive Attention Network for RGB-D Salient Object Detection

TriTransNet: RGB-D Salient Object Detection with a Triplet Transformer Embedding Network

RGB-D Salient Object Detection with Ubiquitous Target Awareness

Contact Info

Product

Resources

About