Disentangled Capsule Routing for Fast Part-Object Relational Saliency

Abstract

Recently, the Part-Object Relational (POR) saliency underpinned by the Capsule Network (CapsNet) has been demonstrated to be an effective modeling mechanism to improve the saliency detection accuracy. However, it is widely known that the current capsule routing operations have huge computational complexity, which seriously limited the usability of the POR saliency models in real-time applications. To this end, this paper takes an early step towards a fast POR saliency inference by proposing a novel disentangled part-object relational network. Concretely, we disentangle horizontal routing and vertical routing from the original omnidirectional capsule routing, thus generating Disentangled Capsule Routing (DCR). This mechanism enjoys two advantages. On one hand, DCR that disentangles orthogonal 1D (i.e., vertical and horizontal) routing greatly reduces parameters and routing complexity, resulting in much faster inference than omnidirectional 2D routing adopted by existing CapsNets. On the other hand, thanks to the light POR cues explored by DCR, we could conveniently integrate the part-object routing process to different feature layers in CNN, rather than just applying it to the small-scaled one as in previous works. This helps to increase saliency inference accuracy. Compared to previous POR saliency detectors, DPORTNet infers visual saliency (5 ∼ 9) × faster, and is more accurate. DPORTNet is available under the open-source license at https://github.com/liuyi1989/DCR.

TCGNet: Type-Correlation Guidance for Salient Object Detection

“…Input size FLOPs (G) Time (s) TSPOANet [20] 352 × 352 197.78 0.32 TSPORTNet [21] 352 × 352 267.50 0.35 POCINet [18] 352 × 352 332.30 0.1 DCR [49] 352 × 352 60.78 0.06 ICON [23] 352 × 352 64.90 0.013 PWHCNet [19] 256 for salient object detection. Our key idea is integrating the correlations of these two cues from CNNs and CapsNets and let them interact.…”

“…On top of that, several efforts are devoted to advocate CapsNets-based part-whole visual saliency [21,48]. To solve the heavy computation of the part-whole relational saliency, Liu et al [49] disentangled the horizontal and vertical capsule routing within the capsule routing algorithm for fast saliency prediction. Besides, a few works have been devoted to the complementary of CNNs and CapsNets.…”

“…To better evaluate the performance of our model, we compare the proposed architecture with 21 state-of-the-art SOD methods, including 14 CNNs based methods (AFNet [28], BASNet [59], PoolNet [60], ASNet [39], PAGE [40], TDBU [41], MINet [15], ITSD [14], F3Net [54], PFSNet [61], LGSL [62], RCSBNet [42]) TRACER [44] and MEMNet [43], 1 Transformer based method (VST [29]), and 6 CapsNet based methods (TSPOANet [68], PWHCNet [19], POCINet [18], DCR [49], ICON [23]).…”

Capsule Networks With Residual Pose Routing

“…3) Performance comparison with state-of-the-arts: Table IX illustrates the quantitative comparison on four benchmarks with 10 state-of-the-art methods, including ICON-S [20], TSPOANet [42], DCR [71], DPNet [72], SelfReformer [73], JointCRF [75], ToHR [76], BMP [77], LFR [78], and Amulet [79]. Specifically in Table IX, ICON-S is one version of ICON [20] with Transformer as the backbone.…”