Hybrid Transformer Based Feature Fusion for Self-Supervised Monocular Depth Estimation

Singh, Tomar, Snehal; Suin, Maitreya; Rajagopalan, A. N.

doi:10.1007/978-3-031-25063-7_19

Cited by 6 publications

(4 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, this makes it possible for some useful feature information to be lost when passing features between each network, thus affecting the quality of the fused features. Therefore, a few methods [27,28] adopt a parallel structure, such as [28], which employs three encoders to obtain helpful information operating at different spatial resolutions, and then integrates these pieces of information using a multi-scale fusion block. However, this approach may suffer from insufficient feature fusion and semantic information.…”

Section: Transformermentioning

confidence: 99%

“…However, the pure Transformer model lacks the ability to model local information due to the absence of spatial inductive bias. To achieve more satisfactory results, some methods have started to combine Transformer with CNNs [13,22,[26][27][28] to leverage the strengths of both approaches. This combination allows for better performance in MDE tasks [13,22,26], as illustrated in Fig.…”

Section: Introductionmentioning

confidence: 99%

“…However, if the features acquired from the previous stage are not accurate enough, it may affect the subsequent features. Alternatively, fewer methods adopt a parallel strategy [27,28], as shown in Fig. 1a, to obtain the last layer of features for fusion through a parallel backbone network to improve the effectiveness and performance of the model.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

PCTDepth: Exploiting Parallel CNNs and Transformer via Dual Attention for Monocular Depth Estimation

Xia,

Duan,

Gao

et al. 2024

Neural Process Lett

View full text Add to dashboard Cite

Monocular depth estimation (MDE) has made great progress with the development of convolutional neural networks (CNNs). However, these approaches suffer from essential shortsightedness due to the utilization of insufficient feature-based reasoning. To this end, we propose an effective parallel CNNs and Transformer model for MDE via dual attention (PCTDepth). Specifically, we use two stream backbones to extract features, where ResNet and Swin Transformer are utilized to obtain local detail features and global long-range dependencies, respectively. Furthermore, a hierarchical fusion module (HFM) is designed to actively exchange beneficial information for the complementation of each representation during the intermediate fusion. Finally, a dual attention module is incorporated for each fused feature in the decoder stage to improve the accuracy of the model by enhancing inter-channel correlations and focusing on relevant spatial locations. Comprehensive experiments on the KITTI dataset demonstrate that the proposed model consistently outperforms the other state-of-the-art methods.

show abstract

Section: Transformermentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

PCTDepth: Exploiting Parallel CNNs and Transformer via Dual Attention for Monocular Depth Estimation

Xia,

Duan,

Gao

et al. 2024

Neural Process Lett

View full text Add to dashboard Cite

show abstract

“…Herein, we call such CNN-Transformer architecture hybrid methods. Soon after, several hybrid models were proposed in the field of MDE [19][20][21].…”

Section: Introductionmentioning

confidence: 99%

Attention Mechanism Used in Monocular Depth Estimation: An Overview

Li,

Wei,

Fan

2023

Applied Sciences

View full text Add to dashboard Cite

Monocular depth estimation (MDE), as one of the fundamental tasks of computer vision, plays important roles in downstream applications such as virtual reality, 3D reconstruction, and robotic navigation. Convolutional neural networks (CNN)-based methods gained remarkable progress compared with traditional methods using visual cues. However, recent researches reveal that the performance of MDE using CNN could be degraded due to the local receptive field of CNN. To bridge the gap, various attention mechanisms were proposed to model the long-range dependency. Although reviews of MDE algorithms based on CNN were reported, a comprehensive outline of how attention boosts MDE performance is not explored yet. In this paper, we firstly categorize recent attention-related works into CNN-based, Transformer-based, and hybrid (CNN–Transformer-based) approaches in the light of how the attention mechanism impacts the extraction of global features. Secondly, we discuss the details and contributions of attention-based MDE methods published from 2020 to 2022. Then, we compare the performance of the typical attention-based methods. Finally, the challenges and trends of the attention mechanism used in MDE are discussed.

show abstract

MDEConvFormer: estimating monocular depth as soft regression based on convolutional transformer

Su,

He,

Zhang

et al. 2024

Multimed Tools Appl

View full text Add to dashboard Cite

Hybrid Transformer Based Feature Fusion for Self-Supervised Monocular Depth Estimation

Cited by 6 publications

References 48 publications

PCTDepth: Exploiting Parallel CNNs and Transformer via Dual Attention for Monocular Depth Estimation

PCTDepth: Exploiting Parallel CNNs and Transformer via Dual Attention for Monocular Depth Estimation

Attention Mechanism Used in Monocular Depth Estimation: An Overview

MDEConvFormer: estimating monocular depth as soft regression based on convolutional transformer

Contact Info

Product

Resources

About