TF-SOD: a novel transformer framework for salient object detection

Abstract

Most of existing saliency object detection 1 models are based on fully convolutional networks (FCNs), 2 which learn multi-scale/level semantic information through 3 convolutional layers to obtain high-quality predicted 4 saliency maps. However, convolution is locally interac-5 tive, and thus it is difficult to capture remote depen-6 dencies. Additionally, FCNs-based methods suffer from 7 coarse object boundaries. In this paper, to solve these 8 problems, we propose a novel transformer framework 9 for salient object detection (named TF-SOD), which 10 consists of the encoder part of the FCN, the fusion mod-11 ule (FM), the transformer module (TM) and the feature 12 decoder module (FDM). Specifically, the FM is a bridge 13 connecting the encoder and TM, and provides some 14 foresight for the non-local interaction of the TM. FDM 15 can efficiently decode the non-local features output by 16 the TM, and achieve deep fusion with local features.17This architecture enables the network to achieve a close 18 integration of local and non-local interactions, making 19 information complementary to each other, deeply min-20 ing the associated information between features. Fur-21 thermore, we also propose a novel edge reinforcement 22 learning strategy, which can effectively suppress edge

Adaptive Dual-Stream Sparse Transformer Network for Salient Object Detection in Optical Remote Sensing Images

“…For instance, a pioneering study by Liu et al [50] presented a unified RGB and RGB-D SOD model based on a vision transformer achieving saliency and boundary detection by introducing task-specific labels. Wang et al [52] proposed a transformer architecture consisting of an FCN decoder and three additional modules to capture salient local and global information in RGB images. The interplay of information from different modalities facilitates the learning of deeper information representations by the network model.…”

“…These works have demonstrated the necessity of replacing CNN with transformer architectures to explore global information in the ORSI-SOD. Moreover, various transformer variants have been developed by researchers for other domains of SOD, resulting in substantial advances in RGB SOD [52], RGB-D/T SOD [53], [54], and Video SOD (VSOD) [55]. However, transformers cannot extract local information as effectively as CNN in Fig.…”

RGB-D Salient Object Detection Based on Cross-Modal and Cross-Level Feature Fusion

“…In the field of computer vision, RGB-D Salient Object Detection (SOD) has progressively evolved into a significant research direction, playing a crucial role in numerous application domains. For instance, in robot navigation [1,2], salient object detection aids robots in gaining a more profound understanding of their environment, thereby informing their decision-making. In the realm of object tracking [3,4], salient object detection can effectively assist the system in accurately locating and tracing objects of interest.…”