Biomedical Image Reconstruction: From the Foundations to Deep Neural Networks

McCann, Michael T.; Unser, Michaël

doi:10.1561/2000000101

Cited by 22 publications

(16 citation statements)

References 104 publications

(126 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, within the context of an end-to-end learned reconstruction mapping, a prior is imposed that is implicitly specified by the distribution of training data and network topology. A comprehensive survey of the current state of deep learning-based methods in tomographic image reconstruction can be found in recent reviews, [9], [29], [30].…”

Section: Regularization In Tomographic Image Reconstructionmentioning

confidence: 99%

On Hallucinations in Tomographic Image Reconstruction

Bhadra

Kelkar

Brooks

et al. 2021

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

Tomographic image reconstruction is generally an ill-posed linear inverse problem. Such ill-posed inverse problems are typically regularized using prior knowledge of the sought-after object property. Recently, deep neural networks have been actively investigated for regularizing image reconstruction problems by learning a prior for the object properties from training images. However, an analysis of the prior information learned by these deep networks and their ability to generalize to data that may lie outside the training distribution is still being explored. An inaccurate prior might lead to false structures being hallucinated in the reconstructed image and that is a cause for serious concern in medical imaging. In this work, we propose to illustrate the effect of the prior imposed by a reconstruction method by decomposing the image estimate into generalized measurement and null components. The concept of a hallucination map is introduced for the general purpose of understanding the effect of the prior in regularized reconstruction methods. Numerical studies are conducted corresponding to a stylized tomographic imaging modality. The behavior of different reconstruction methods under the proposed formalism is discussed with the help of the numerical studies.

show abstract

Section: Regularization In Tomographic Image Reconstructionmentioning

confidence: 99%

On Hallucinations in Tomographic Image Reconstruction

Bhadra

Kelkar

Brooks

et al. 2021

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

show abstract

“…To solve x, a general approach that covers many applications in image and signal processing [47]- [50], [53]- [55] involves minimizing a convex optimization problem of the form minimize…”

Section: B Image Reconstruction Using Mathematical Optimizationmentioning

confidence: 99%

“…Equation ( 9) can be solved by several techniques, including the alternating direction method of multipliers (ADMM) [52], [55], and accelerated proximal gradient (APG) [27], [47]- [50], [53]- [55]. Mathematical details of these minimization algorithms, as used in our approach, are provided in Appendix A.…”

Section: B Image Reconstruction Using Mathematical Optimizationmentioning

confidence: 99%

Opto-Acoustic Image Reconstruction and Motion Tracking Using Convex Optimization

Zalev¹,

Kolios²

2021

IEEE Trans. Comput. Imaging

View full text Add to dashboard Cite

Opto-acoustic imaging systems detect acoustic waves produced by optical absorption to visualize molecular contrast in biological tissue. This permits non-invasive vascular assessment of benign and malignant tumors. In this article, we describe a framework to iteratively determine the motion of an opto-acoustic probe during a minimization-based image reconstruction process. The probe emits light and uses an ultrasonic transducer array to acquire data for cross-sectional slices of tissue. To improve visibility, our technique uses multiple 2D slices to perform 3D volumetric reconstruction. Our model includes wavelength-specific optical absorption, position-dependent illumination and a realistic transducer element geometry. We investigate this technique using simulated, experimentally collected, and clinically acquired data. By performing 3D image reconstruction on a digital phantom, we demonstrate estimation of elevational probe motion without external sensors. We compare images of a benign lesion from a clinical breast imaging study and observe significant artifact reduction and contrast-to-background ratio improvement using our technique. The approach has potential to improve opto-acoustic image visibility for assessment of breast cancer or other diseases.

show abstract

“…Note that the fully connected layer, used to back project the sensor data in the image domain, non-linearly increases the memory requirements of the neural network, thereby limiting the maximum number of additional convolution layers. This layer is deemed compulsory for an inverse scattering problem where direct analytical reconstruction is not well defined, unlike in magnetic resonance imaging or computed tomography where a well-defined inversion exists using Fourier transform or Radon transform [36].…”

Section: B Network Depth Trade-off: Complexity Versus Performancementioning

confidence: 99%

Multitask Deep Learning Reconstruction and Localization of Lesions in Limited Angle Diffuse Optical Tomography

Yedder¹,

Cardoen²,

Hamarneh³

2020

Preprint

View full text Add to dashboard Cite

<div>Diffuse optical tomography (DOT) leverages near-infrared light propagation through tissue to assess its optical properties and identify abnormalities. </div><div>DOT image reconstruction from limited-angle data acquisition is severely ill-posed due to the highly scattered photons in the medium and the relatively small number of collected projections. Reconstructions are thus commonly marred by artifacts and, as a result, it is difficult to obtain accurate reconstruction of target objects, e.g., malignant lesions.</div><div>Reconstruction does not always ensure good localization of small lesions. Furthermore, conventional optimization-based reconstruction methods are computationally expensive, rendering them too slow for real-time imaging applications.</div><div>Our goal is to develop a fast and accurate image reconstruction method using deep learning, where multitask learning ensures accurate lesion localization in addition to improved reconstruction.</div><div>We apply spatial-wise attention and a distance transform based loss function in a novel multitask learning formulation to improve localization and reconstruction compared to single-task optimized methods.</div><div>Given the scarcity of real-world sensor-image pairs required for training supervised deep learning models,</div><div>we leverage physics-based simulation to generate synthetic datasets and use a transfer learning module to align the sensor domain distribution between in silico and real-world data, while taking advantage of cross-domain learning.</div><div>Both quantitative and qualitative results on phantom and real data indicate the superiority of our multitask method in the reconstruction and localization of lesions in tissue compared to state-of-the-art methods.</div><div>The results demonstrate that multitask learning provides sharper and more accurate reconstruction.</div>

show abstract

Biomedical Image Reconstruction: From the Foundations to Deep Neural Networks

Abstract: RKHS reproducing kernel Hilbert spaces SGD stochastic gradient descent SIM structured-illumination microscopy SNR signal-to-noise ratio SPECT single-photon emission computed tomography SSIM structural similarity index TCIA The Cancer Imaging Archive TV total variation

Cited by 22 publications

References 104 publications

On Hallucinations in Tomographic Image Reconstruction

On Hallucinations in Tomographic Image Reconstruction

Opto-Acoustic Image Reconstruction and Motion Tracking Using Convex Optimization

Multitask Deep Learning Reconstruction and Localization of Lesions in Limited Angle Diffuse Optical Tomography

Contact Info

Product

Resources

About