Material Decomposition in Spectral CT Using Deep Learning: A Sim2Real Transfer Approach

Abascal, Juan F P J; Ducros, Nicolas; Pronina, Valeriya; Rit, Simon; Rodesch, Pierre‐Antoine; Broussaud, Thomas; Bussod, Suzanne; Douek, Philippe; Hauptmann, Andreas; Arridge, Simon; Peyrin, Françoise

doi:10.1109/access.2021.3056150

Cited by 26 publications

(20 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A more detailed survey is given in [53]. Apart from robotics tasks, sim2real methods have also been widely used in other fields including autonomous driving [51,34], medical diagnosis [1], or even the control of atmospheric pressure plasma jets [48].…”

Section: Related Workmentioning

confidence: 99%

Sim2real Transfer Learning for Point Cloud Segmentation: An Industrial Application Case on Autonomous Disassembly

Xuelei

Pfrommer

et al. 2023

2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

On robotics computer vision tasks, generating and annotating large amounts of data from real-world for the use of deep learning-based approaches is often difficult or even impossible. A common strategy for solving this problem is to apply simulation-to-reality (sim2real) approaches with the help of simulated scenes. While the majority of current robotics vision sim2real work focuses on image data, we present an industrial application case that uses sim2real transfer learning for point cloud data. We provide insights on how to generate and process synthetic point cloud data in order to achieve better performance when the learned model is transferred to real-world data. The issue of imbalanced learning is investigated using multiple strategies. A novel patch-based attention network is proposed additionally to tackle this problem.

show abstract

Section: Related Workmentioning

confidence: 99%

Sim2real Transfer Learning for Point Cloud Segmentation: An Industrial Application Case on Autonomous Disassembly

Xuelei

Pfrommer

et al. 2023

2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

show abstract

“…Still, degraded spatial resolution is a concern, 17,21 and the regularization may affect the accuracy when the noise level is substantial 22 . Recently, learning‐based material decomposition has been an active research field, showing promising results with superior performance compared to conventional methods 23–26 . However, the robustness of the data mismatch needs to be addressed, and a sufficient number of training data is not always available.…”

Section: Introductionmentioning

confidence: 99%

Likelihood‐based bilateral filters for pre‐estimated basis sinograms using photon‐counting CT

Lee

2023

Medical Physics

View full text Add to dashboard Cite

Background Noise amplification in material decomposition is an issue for exploiting photon‐counting computed tomography (PCCT). Regularization techniques and neighborhood filters have been widely used, but degraded spatial resolution and bias are concerns. Purpose This paper proposes likelihood‐based bilateral filters that can be applied to pre‐estimated basis sinograms to reduce the noise while minimally affecting spatial resolution and accuracy. Methods The proposed method needs system models (e.g., incident spectrum, detector response) to calculate the likelihood. First, it performs maximum likelihood (ML)‐based estimation in the projection domain to obtain basis sinograms. The estimated basis sinograms suffer from severe noise but are asymptotically unbiased without degrading spatial resolution. Then it calculates the neighborhood likelihoods for a given measurement at the center pixel using the neighborhood estimates and designs the weights based on the distance of likelihoods. It is also analyzed in terms of statistical inference, and then two variations of the filter are introduced: one that requires the significance level instead of the empirical hyperparameter. The other is a measurement‐based filter, which can be applied when accurate estimates are given without the system models. The proposed methods were validated by analyzing the local property of noise and spatial resolution and the global trend of noise and bias using numerical thorax and abdominal phantoms for a two‐material decomposition (water and bone). They were compared to the conventional neighborhood filters and the model‐based iterative reconstruction with an edge‐preserving penalty applied in the basis images. Results The proposed method showed comparable or superior performance for the local and global properties to conventional methods in many cases. The thorax phantom: The full width at half maximum (FWHM) decreased by −2%–31% (−2 indicates that it increased by 2% compared to the best performance from conventional methods), and the global bias was reduced by 2%–19% compared to other methods for similar noise levels (local: 51% of the ML, global: 49%) in the water basis image. The FWHM decreased by 8%–31%, and the global bias was reduced by 9%–44% for similar noise levels (local: 44% of the ML, global: 36%) in the CT image at 65 keV. The abdominal phantom: The FWHM decreased by 10%–32%, and the global bias was reduced by 3%–35% compared to other methods for similar noise levels (local: 66% of the ML, global: 67%) in the water basis image. The FWHM decreased by up to −11%–47%, and the global bias was reduced by 13%–35% for similar noise levels (local: 71% of the ML, global: 70%) in the CT image at 60 keV. Conclusions This paper introduced the likelihood‐based bilateral filters as a post‐processing method applied to the ML‐based estimates of basis sinograms. The proposed filters effectively reduced the noise in the basis images and the synthesized monochromatic CT images. It showed the potential of using likelihood‐based filters ...

show abstract

“…Lots of MMD methods for DECT imaging have been developed [5][6][7][8][9][10][11][12] , and they can be divided into two classes, which are traditional MMD methods [5][6][7][8][9] and deep learning (DL)-based MMD methods [9][10][11][12][13] . In the traditional methods, Long et al proposed a flexible MMD method to accurately estimate volume fractions by looping over all possibility of materials under the non-negativity constraint 5 .…”

Section: Introductionmentioning

confidence: 99%

“…Shi et al proposed a DIWGAN network with two interactive generators to estimate two material-specific images from DECT 12 . Juan et al combined simulated and real data using transfer learning network to improve network performance 13 .…”

Section: Introductionmentioning

confidence: 99%

Self-attention network for weak-supervised learning multi-material decomposition in dual energy CT

Duan

Liao

et al. 2023

Medical Imaging 2023: Physics of Medical Imaging

View full text Add to dashboard Cite

Inspired by the deep learning techniques, data-driven methods have been developed to promote image quality and material decomposition accuracy in dual energy computed tomography (DECT) imaging. Most of these data-driven DECT imaging methods exploit the image priors within large amount of training data to learn the mapping function from the noisy DECT images to the desired high-quality material images in a supervised manner. Meanwhile, these supervised DECT imaging methods only estimate the multiple material images directly from the network but the material decomposition mechanism is not included in the network, and they fail to consider the unlabeled noisy DECT images to further improve the performance. In this work, to address these issues, we propose a novel Weak-supervised learning Multi-material Decomposition Network with self-attention mechanism (WMD-Net) to estimate multiple material images from DECT images with the combination of labeled and unlabeled DECT images accurately and effectively. Specifically, in the proposed WMD-Net, the labeled DECT images are used to estimate the three material images in a supervised sub-network, and the unlabeled DECT images are used to construct the unsupervised sub-network with the benefit of material decomposition mechanism. Finally, the two sub-networks are introduced into the proposed WMD-Net method. The proposed WMD-Net method is validated and evaluated through the synthesized clinical data, and the experimental results demonstrate that the proposed WMD-Net method can estimate more accurate material images than the other competing methods in terms of noise-induced artifacts reduction and structure details preservation.

show abstract

Material Decomposition in Spectral CT Using Deep Learning: A Sim2Real Transfer Approach

Cited by 26 publications

References 54 publications

Sim2real Transfer Learning for Point Cloud Segmentation: An Industrial Application Case on Autonomous Disassembly

Sim2real Transfer Learning for Point Cloud Segmentation: An Industrial Application Case on Autonomous Disassembly

Likelihood‐based bilateral filters for pre‐estimated basis sinograms using photon‐counting CT

Self-attention network for weak-supervised learning multi-material decomposition in dual energy CT

Contact Info

Product

Resources

About