Bootstrapped CNNs for Building Segmentation on RGB-D Aerial Imagery

Sebastian, Clint; Boom, Bas; Lankveld, Thijs van; Bondarev, Egor; With, Peter H. N. de

doi:10.48550/arxiv.1810.03570

Cited by 1 publication

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This process is critical for characterizing many medical conditions, where a diagnostic workup may depend on information from patient history, physical examination, organ-level medical imaging, histological analysis, and laboratory studies. Even in a single medical imaging study, physicians commonly rely on multimodal contrast to determine an optimal diagnosis, for instance, considering both T1-and T2-weighted MRI images [9,13] or concatenation [29,33]. We propose using a densenet style architecture and fusing the multimodal information in the last eight layers of the first dense block, hence harnessing the benefits of both multi-layer fusion and densely connected networks.…”

Section: Introductionmentioning

confidence: 99%

“…A comparison of commonly used multimodal deep learning architectures with the proposed architecture. The state-of-theart methods involve multi-layer fusion[9,13] or concatenation[29,33]. We propose using a densenet style architecture and fusing the multimodal information in the last eight layers of the first dense block, hence harnessing the benefits of both multi-layer fusion and densely connected networks.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Multimodal Densenet

Mahmood¹,

Yang²,

Ashley³

et al. 2018

Preprint

View full text Add to dashboard Cite

Humans make accurate decisions by interpreting complex data from multiple sources. Medical diagnostics, in particular, often hinge on human interpretation of multimodal information. In order for artificial intelligence to make progress in automated, objective, and accurate diagnosis and prognosis, methods to fuse information from multiple medical imaging modalities are required. However, combining information from multiple data sources has several challenges, as current deep learning architectures lack the ability to extract useful representations from multimodal information, and often simple concatenation is used to fuse such information. In this work, we propose Multimodal DenseNet, a novel architecture for fusing multimodal data. Instead of focusing on concatenation or early and late fusion, our proposed architectures fuses information over several layers and gives the model flexibility in how it combines information from multiple sources. We apply this architecture to the challenge of polyp characterization and landmark identification in endoscopy. Features from white light images are fused with features from narrow band imaging or depth maps. This study demonstrates that Multimodal DenseNet outperforms monomodal classification as well as other multimodal fusion techniques by a significant margin on two different datasets.

show abstract