More Diverse Means Better: Multimodal Deep Learning Meets Remote-Sensing Imagery Classification

Hong, Danfeng; Gao, Lianru; Yokoya, Naoto; Yao, Jing; Chanussot, Jocelyn; Du, Qian; Zhang, Bing

doi:10.1109/tgrs.2020.3016820

Cited by 942 publications

(380 citation statements)

References 54 publications

Supporting

Mentioning

377

Contrasting

Order By: Relevance

“…I Nrecent years, with the rapid development of remote sensing technology, remote sensing images' spatial resolution has gradually increased, which makes the demand for automatic interpretation technology of remote sensing images increasing. Land cover classification [37][38][39][40][41][42], scene-level geographic image classification [36], and geospatial object detection and recognition [34,35] are basic and challenging research contents in the automatic interpretation of remote sensing images and have caused a wide range of attention and research.…”

Section: Introductionmentioning

confidence: 99%

SRARNet: A Unified Framework for Joint Superresolution and Aircraft Recognition

Tang

Deng

Han

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Section: Introductionmentioning

confidence: 99%

SRARNet: A Unified Framework for Joint Superresolution and Aircraft Recognition

Tang

Deng

Han

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

“…8(A), we produced the gradient field and K-means clustering images as two new modalities for extracting shallow features. There are many methods to fuse multimodal inputs, and concatenate-based fusion is an intuitive fusion method [32], but this method is more suitable for situations where each modality is equally important for classification. Extracting features first and then concatenating is also a very popular fusion method [33], while the number of parameters and GPU memory limit its application.…”

Section: Multimodal Generation and Fusionmentioning

confidence: 99%

Evaluation of Lung Involvement in COVID-19 Pneumonia Based on Ultrasound Images

Liu

Dong

et al. 2020

Preprint

View full text Add to dashboard Cite

Background: Lung ultrasound (LUS) can be an important imaging tool for the diagnosis and assessment of lung involvement. Ultrasound sonograms have been confirmed to illustrate damage to a person’s lungs, which means that the correct classification and scoring of a patient’s sonogram can be used to assess lung involvement.Methods: The purpose of this study was to establish a lung involvement assessment model based on deep learning. A novel multimodal channel and receptive field attention network combined with ResNeXt (MCRFNet) was proposed to classify sonograms, and the network can automatically fuse shallow features and determine the importance of different channels and respective fields. Finally, sonogram classes were transformed into scores to evaluate lung involvement from the initial diagnosis to rehabilitation.Results and conclusion: Using multicenter and multimodal ultrasound data from 104 patients, the diagnostic model achieved 94.39% accuracy, 82.28% precision, 76.27% sensitivity, and 96.44% specificity. The lung involvement severity and the trend of COVID-19 pneumonia were evaluated quantitatively.

show abstract

“…Different from shallow machine learning methods, the deep network structure of deep learning has better feature mining performance. Therefore, many researchers have applied CNN to HSIC and demonstrated that CNN can show exceedingly promising performance [24][25][26][27][28][29][30][31][32][33][34][35][36]. For example, in [24], the spatial and spectral features of the original HSI are simultaneously extracted by 3D-CNN.…”

Section: Introductionmentioning

confidence: 99%

Sandwich Convolutional Neural Network for Hyperspectral Image Classification Using Spectral Feature Enhancement

Chen

2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Recently, convolutional neural networks (CNNs) has been used to extract spectral and spatial features of hyperspectral images (HSIs) for hyperspectral image classification (HSIC) because of their excellent performance in extracting and analyzing complex data. However, due to the limited labeled samples and existing mixed pixels, it is difficult to extract features effectively, which further leads to the problem of overfitting of the model. On the other hand, to improve the extraction ability of the CNN, the depth of the model, and the complexity of the convolution kernel often need to be increased. In this paper, a sandwich CNN based on spectral feature enhancement (SFE-SCNN) is proposed for HSIC. The proposed method, SFE-SCNN, introduces the spectral feature enhancement operation, which makes the data reflect more discriminative spectral feature details to suppress the interference of mixed pixels. Furthermore, according to the preprocessed data structure features, a lightweight sandwich convolution neural network is proposed. To fully extract the spectral features, the spectral feature re-extraction operation is used for the first time. Experimental results on three real hyperspectral data sets demonstrate that the proposed method achieves better classification performance than other state-of-the-art methods.

show abstract

More Diverse Means Better: Multimodal Deep Learning Meets Remote-Sensing Imagery Classification

Cited by 942 publications

References 54 publications

SRARNet: A Unified Framework for Joint Superresolution and Aircraft Recognition

SRARNet: A Unified Framework for Joint Superresolution and Aircraft Recognition

Evaluation of Lung Involvement in COVID-19 Pneumonia Based on Ultrasound Images

Sandwich Convolutional Neural Network for Hyperspectral Image Classification Using Spectral Feature Enhancement

Contact Info

Product

Resources

About