Multi-Content Complementation Network for Salient Object Detection in Optical Remote Sensing Images

Abstract

Salient object detection (SOD) in optical remote sensing images (RSIs), or RSI-SOD, is an emerging topic in understanding optical RSIs. However, due to the difference between optical RSIs and natural scene images (NSIs), directly applying NSI-SOD methods to optical RSIs fails to achieve satisfactory results. In this paper, we propose a novel Adjacent Context Coordination Network (ACCoNet) to explore the coordination of adjacent features in an encoder-decoder architecture for RSI-SOD. Specifically, ACCoNet consists of three parts: an encoder, Adjacent Context Coordination Modules (ACCoMs), and a decoder. As the key component of ACCoNet, ACCoM activates the salient regions of output features of the encoder and transmits them to the decoder. ACCoM contains a local branch and two adjacent branches to coordinate the multilevel features simultaneously. The local branch highlights the salient regions in an adaptive way, while the adjacent branches introduce global information of adjacent levels to enhance salient regions. Additionally, to extend the capabilities of the classic decoder block (i.e., several cascaded convolutional layers), we extend it with two bifurcations and propose a Bifurcation-Aggregation Block to capture the contextual information in the decoder. Extensive experiments on two benchmark datasets demonstrate that the proposed ACCoNet outperforms 22 state-ofthe-art methods under nine evaluation metrics, and runs up to 81 fps on a single NVIDIA Titan X GPU. The code and results of our method are available at https://github.com/MathLee/ACCoNet.

Lightweight Salient Object Detection in Optical Remote-Sensing Images via Semantic Matching and Edge Alignment

“…Tu et al [39] and Zhou et al [22], following [19], extracted boundary information based on low-level and high-level features to preserve boundaries of salient objects in two decoders. Li et al [14] integrated edge with foreground, background, and global information, and took full account of the complementarity between these information to adapt to ORSIs.…”

“…We propose ESAM to effectively and efficiently explore edge information for detail enhancement to preserve the complex shapes of salient objects in ORSIs. Different from some edge-based ORSI-SOD methods [14], [22], [38], [39], our ESAM is lightweight, and extracts edge information without using edge supervision, which is more convenient. Like DSMM, ESAM also fully considers the spatial interaction and channel interaction of features.…”

“…Speed #Param FLOPs EORSSD [38] ORSSD [21] size SARNet [35], MJRBM [39], EMFINet [22], and MCC-Net [14]). In addition, we also compare our lightweight SeaNet with 4 state-of-the-art lightweight SOD methods, including 3 lightweight NSI-SOD methods (i.e., CSNet [40], SAM-Net [41], and HVPNet [13]), and the lightweight ORSI-SOD method CorrNet [15].…”

“…Saliency maps produced by four types of methods and our method on ORSIs. PA-KRN [12] is an NSI-SOD method, HVPNet [13] is a lightweight NSI-SOD method, MCCNet [14] is an ORSI-SOD method, and CorrNet [15] is a lightweight ORSI-SOD method. Please zoom-in for details.…”

“…For example, LVNet [21] and EMFINet [22] take ORSIs with multiple resolutions as inputs to overcome the problem of variable sizes of salient objects. MCCNet [14] comprehensively integrates foreground, background, edge, and global information to deal with the complex background of ORSIs, producing good saliency maps as shown in the fifth column of Fig. 1.…”