An End-To-End Bayesian Segmentation Network Based on a Generative Adversarial Network for Remote Sensing Images

Xiong, Dehui; He, Chu; Liu, Xinlong; Liao, Mingsheng

doi:10.3390/rs12020216

Cited by 18 publications

(3 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nearly all applications of Bayesian deep learning (BDL) methods in the literature arise in medical imaging or autonomous driving domains [5,6]. Only one paper has explored BDL in the SAR domain [31]. However, this paper only achieves a maximum a posteriori (MAP) point estimate of uncertainty, as their generative adversarial network (GAN)-based approach is deterministic: it takes a previous network's outputs as its inputs, as opposed to the Gaussian noise that a standard GAN receives.…”

Section: Prior Workmentioning

confidence: 99%

Uncertainty Estimation for Deep Learning-Based Segmentation of Roads in Synthetic Aperture Radar Imagery

Haas

Rabus

2021

Remote Sensing

View full text Add to dashboard Cite

Mission-critical applications that rely on deep learning (DL) for automation suffer because DL models struggle to provide reliable indicators of failure. Reliable failure prediction can greatly improve the efficiency of a system, because it becomes easier to predict when human intervention is required. DL-based systems thus stand to benefit greatly from robust measures of uncertainty over model predictions. Monte Carlo dropout (MCD), a Bayesian method, and deep ensembles (DE) have emerged as two of the most popular and competitive ways to perform uncertainty estimation. Although literature exploring the usefulness of these approaches exists in medical imaging, robotics and autonomous driving domains, it is scarce to non-existent for remote sensing, and in particular, synthetic aperture radar (SAR) applications. To close this gap, we have created a deep learning model for road extraction (hereafter referred to as segmentation) in SAR and use it to compare standard model outputs against the aforementioned most popular methods for uncertainty estimation, MCD and DE. We demonstrate that these methods are not effective as an indicator of segmentation quality when measuring uncertainty (as indicated by model softmax outputs) across an entire image but are effective when uncertainty is measured from the set of road predictions only. Furthermore, we show a marked improvement in the correlation between prediction uncertainty and segmentation quality when we increase the set of road predictions by including predictions with lower softmax scores. We demonstrate the efficacy of our application of MCD and DE methods with an experimental design that measures performance in real-world quality assessment using in-distribution (ID) and out-of-distribution (OOD) data. These results inform the development of mission-critical deep learning systems in remote sensing. Tasks in medical image analysis that have a similar morphology to road structures, such as blood vessel segmentation, can also benefit from our findings.

show abstract

Section: Prior Workmentioning

confidence: 99%

Uncertainty Estimation for Deep Learning-Based Segmentation of Roads in Synthetic Aperture Radar Imagery

Haas

Rabus

2021

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…In recent years, a range of dehazing methods have been developed, roughly divided into prior and deep learning-based methods [7,8]. Most prior-based methods are based on the atmospheric scattering model and require accurate estimation for the transmission and atmospheric light [9].…”

Section: Introductionmentioning

confidence: 99%

Hierarchical Semantic-Guided Contextual Structure-Aware Network for Satellite Image Dehazing

Yang,

Cao,

Wang

et al. 2024

Preprint

View full text Add to dashboard Cite

Haze always shrouds satellite images, obscuring valuable geographic information for military surveillance, natural calamity surveillance and mineral resource exploration. Satellite image dehazing (SID) provides the possibility for better applications of satellite images. Most of the existing dehazing methods are tailored for natural images and are not very effective for satellite images with non-homogeneous haze since the semantic structure information and inconsistent attenuation are not fully considered. To tackle this problem, this study proposes a hierarchical semantic-guided contextual structure-aware network (SCSNet) for satellite image dehazing. Specifically, a hybrid CNN-transformer architecture integrated with a hierarchical semantic guidance (HSG) module is presented to learn semantic structure information by synergetically complementing local representation from non-local features. Furthermore, a cross-layer fusion (CLF) module is specially designed to replace the traditional skip connection during the feature decoding stage, so as to reinforce the attention to the spatial regions and feature channels with more serious attenuation. The results on the SateHaze1k, RS-Haze, and RSID datasets demonstrate that the proposed SCSNet can achieve effective dehazing and outperforms existing state-of-the-art methods.

show abstract

“…Deep learning is useful for automating various tasks in radiation oncology, most notably organ segmentation but has also been used for dose calculations and linear accelerator quality assurance [14][15][16][17][18][19][20][21]. Generative deep learning models such as generative adversarial networks have been used in other domains to perform image-to-image translation tasks such as: converting photographs taken during the day to night, converting hand-drawn pictures into realistic photos, and converting satellite images into maps [22][23][24][25][26][27]. They have also been used in medical applications to perform tasks such as converting MRI datasets into synthetic CT datasets [28][29][30][31][32][33][34].…”

Section: Introductionmentioning

confidence: 99%

Synthetic cranial MRI from 3D optical surface scans using deep learning for radiation therapy treatment planning

et al. 2023

View full text Add to dashboard Cite

Background Optical scanning technologies are increasingly being utilised to supplement treatment workflows in radiation oncology, such as surface-guided radiotherapy or 3D printing custom bolus. One limitation of optical scanning devices is the absence of internal anatomical information of the patient being scanned. As a result, conventional radiation therapy treatment planning using this imaging modality is not feasible. Deep learning is useful for automating various manual tasks in radiation oncology, most notably, organ segmentation and treatment planning. Deep learning models have also been used to transform MRI datasets into synthetic CT datasets, facilitating the development of MRI-only radiation therapy planning. Aims To train a pix2pix generative adversarial network to transform 3D optical scan data into estimated MRI datasets for a given patient to provide additional anatomical data for a select few radiation therapy treatment sites. The proposed network may provide useful anatomical information for treatment planning of surface mould brachytherapy, total body irradiation, and total skin electron therapy, for example, without delivering any imaging dose. Methods A 2D pix2pix GAN was trained on 15,000 axial MRI slices of healthy adult brains paired with corresponding external mask slices. The model was validated on a further 5000 previously unseen external mask slices. The predictions were compared with the “ground-truth” MRI slices using the multi-scale structural similarity index (MSSI) metric. A certified neuro-radiologist was subsequently consulted to provide an independent review of the model’s performance in terms of anatomical accuracy and consistency. The network was then applied to a 3D photogrammetry scan of a test subject to demonstrate the feasibility of this novel technique. Results The trained pix2pix network predicted MRI slices with a mean MSSI of 0.831 ± 0.057 for the 5000 validation images indicating that it is possible to estimate a significant proportion of a patient’s gross cranial anatomy from a patient’s exterior contour. When independently reviewed by a certified neuro-radiologist, the model’s performance was described as “quite amazing, but there are limitations in the regions where there is wide variation within the normal population.” When the trained network was applied to a 3D model of a human subject acquired using optical photogrammetry, the network could estimate the corresponding MRI volume for that subject with good qualitative accuracy. However, a ground-truth MRI baseline was not available for quantitative comparison. Conclusions A deep learning model was developed, to transform 3D optical scan data of a patient into an estimated MRI volume, potentially increasing the usefulness of optical scanning in radiation therapy planning. This work has demonstrated that much of the human cranial anatomy can be predicted from the external shape of the head and may provide an additional source of valuable imaging data. Further research is required to investigate the feasibility of this approach for use in a clinical setting and further improve the model’s accuracy.

show abstract

An End-To-End Bayesian Segmentation Network Based on a Generative Adversarial Network for Remote Sensing Images

Cited by 18 publications

References 33 publications

Uncertainty Estimation for Deep Learning-Based Segmentation of Roads in Synthetic Aperture Radar Imagery

Uncertainty Estimation for Deep Learning-Based Segmentation of Roads in Synthetic Aperture Radar Imagery

Hierarchical Semantic-Guided Contextual Structure-Aware Network for Satellite Image Dehazing

Synthetic cranial MRI from 3D optical surface scans using deep learning for radiation therapy treatment planning

Contact Info

Product

Resources

About