Exemplar-based inpainting as a solution to the missing wedge problem in electron tomography

Trampert, Patrick; Wu, Wei; Chen, Delei; Ravelli, Raimond B. G.; Dahmen, Tim; Peters, Peter J.; Kübel, Christian; Slusallek, Philipp

doi:10.1016/j.ultramic.2018.04.001

Cited by 12 publications

(7 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Missing wedge restoration based on a Monte Carlo evaluation of randomly introduced noise can extend some information into missing Fourier components based on consistency with the sampled data (41). Finally, inpainting and deep learning approaches begin to emerge for optimization of tomogram reconstruction (42,43), annotation (44), and particle extraction (45). Deconvolution is a deterministic algorithm that exploits prior knowledge about the 3D representation of a point object.…”

Section: Discussionmentioning

confidence: 99%

Three-dimensional deconvolution processing for STEM cryotomography

Waugh

Wolf

Fass

et al. 2020

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

The complex environment of biological cells and tissues has motivated development of three-dimensional (3D) imaging in both light and electron microscopies. To this end, one of the primary tools in fluorescence microscopy is that of computational deconvolution. Wide-field fluorescence images are often corrupted by haze due to out-of-focus light, i.e., to cross-talk between different object planes as represented in the 3D image. Using prior understanding of the image formation mechanism, it is possible to suppress the cross-talk and reassign the unfocused light to its proper source post facto. Electron tomography based on tilted projections also exhibits a cross-talk between distant planes due to the discrete angular sampling and limited tilt range. By use of a suitably synthesized 3D point spread function, we show here that deconvolution leads to similar improvements in volume data reconstructed from cryoscanning transmission electron tomography (CSTET), namely a dramatic in-plane noise reduction and improved representation of features in the axial dimension. Contrast enhancement is demonstrated first with colloidal gold particles and then in representative cryotomograms of intact cells. Deconvolution of CSTET data collected from the periphery of an intact nucleus revealed partially condensed, extended structures in interphase chromatin.

show abstract

Section: Discussionmentioning

confidence: 99%

Three-dimensional deconvolution processing for STEM cryotomography

Waugh

Wolf

Fass

et al. 2020

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

show abstract

“…Missing Wedge Restoration based on a Monte Carlo evaluation of randomly introduced noise can extend some information into missing Fourier components based on consistency with the sampled data (42). Finally, inpainting and deep learning approaches begin to emerge for optimization of tomogram reconstruction (43,44), annotation (45), and particle extraction (46). Deconvolution is a deterministic algorithm that exploits prior knowledge about the 3D representation of a point object.…”

Section: Discussionmentioning

confidence: 99%

3D Deconvolution Processing for STEM Cryo-Tomography

Waugh

Wolf

Fass

et al. 2020

Preprint

View full text Add to dashboard Cite

The complex environment of biological cells and tissues has motivated development of three dimensional imaging in both light and electron microscopies. To this end, one of the primary tools in fluorescence microscopy is that of computational deconvolution. Wide-field fluorescence images are often corrupted by haze due to out-of-focus light, i.e., to cross-talk between different object planes as represented in the 3D image. Using prior understanding of the image formation mechanism, it is possible to suppress the cross-talk and reassign the unfocused light to its proper source post facto. Electron tomography based on tilted projections also exhibits a cross-talk between distant planes due to the discrete angular sampling and limited tilt range. By use of a suitably synthesized 3D point spread function, we show here that deconvolution leads to similar improvements in volume data reconstructed from cryo-scanning transmission electron tomography (CSTET), namely a dramatic in-plane noise reduction and improved representation of features in the axial dimension. Contrast enhancement is demonstrated first with colloidal gold particles, and then in representative cryo-tomograms of intact cells. Deconvolution of CSTET data collected from the periphery of an intact nucleus revealed partially condensed, extended structures in interphase chromatin.Significance statementElectron tomography is used to reveal the structure of cells in three dimensions. The combination with cryogenic fixation provides a snapshot in time of the living state. However, cryo-tomography normally requires very thin specimens due to image formation by conventional phase contrast transmission electron microscopy (TEM). The thickness constraint can be relaxed considerably by scanning TEM (STEM), yet three-dimensional (3D) reconstruction is still subject to artifacts inherent in the collection of data by tilted projections. We show here that deconvolution algorithms developed for fluorescence microscopy can suppress these artifacts, resulting in significant contrast enhancement. The method is demonstrated by cellular tomography of complex membrane structures, and by segmentation of chromatin into distinct, contiguous domains of heterochromatin and euchromatin at high and low density, respectively.

show abstract

“…Trampert et al . (2018a, b) and Trampert (2019) extend the exemplar‐based inpainting techniques to three dimensional EM data. Tran & Tran (2019) propose a reconstruction framework based on three different nonlearning image inpainting algorithms to solve the microstructure reconstruction problem in three contexts with different textures.…”

Section: Introductionmentioning

confidence: 99%

Deep learning‐based automatic inpainting for material microscopic images

Ma²,

Gao

et al. 2020

Journal of Microscopy

View full text Add to dashboard Cite

Summary The microscopic image is important data for recording the microstructure information of materials. Researchers usually use image‐processing algorithms to extract material features from that and then characterise the material microstructure. However, the microscopic images obtained by a microscope often have random damaged regions, which will cause the loss of information and thus inevitably influence the accuracy of microstructural characterisation, even lead to a wrong result. To handle this problem, we provide a deep learning‐based fully automatic method for detecting and inpainting damaged regions in material microscopic images, which can automatically inpaint damaged regions with different positions and shapes, as well as we also use a data augmentation method to improve the performance of inpainting model. We evaluate our method on Al–La alloy microscopic images, which indicates that our method can achieve promising performance on inpainted and material microstructure characterisation results compared to other image inpainting software for both accuracy and time consumption. Lay Description A basic goal of materials data analysis is to extract useful information from materials datasets that can in turn be used to establish connections along the composition–processing–structure–properties chain. The microscopic images obtained by a microscope is the key carrier of material microstructural information. Researchers usually use image analysis algorithms to extract regions of interest or useful features from microscopic images, aiming to analyse material microstructure, organ tissues or device quality etc. Therefore, the integrity and clarity of the microscopic image are the most important attributes for image feature extraction. Scientists and engineers have been trying to develop various technologies to obtain perfect microscopic images. However, in practice, some extrinsic defects are often introduced during the preparation and/or shooting processes, and the elimination of these defects often requires mass efforts and cost, or even is impossible at present. Take the microstructure image of metallic material for example, samples prepared to microstructure characterisation often need to go through several steps such as cutting, grinding with sandpaper, polishing, etching, and cleaning. During the grinding and polishing process, defects such as scratches could be introduced. During the etching and cleaning process, some defects such as rust caused by substandard etching, stains etc. may arise and be persisted. These defects can be treated as damaged regions with nonfixed positions, different sizes, and random shapes, resulting in the loss of information, which seriously affects subsequent visual observation and microstructural feature extraction. To handle this problem, we provide a deep learning‐based fully automatic method for detecting and inpainting damaged regions in material microscopic images, which can automatically inpaint damaged regions with different positions and shapes, as well as we also us...

show abstract

Exemplar-based inpainting as a solution to the missing wedge problem in electron tomography

Cited by 12 publications

References 29 publications

Three-dimensional deconvolution processing for STEM cryotomography

Three-dimensional deconvolution processing for STEM cryotomography

3D Deconvolution Processing for STEM Cryo-Tomography

Deep learning‐based automatic inpainting for material microscopic images

Contact Info

Product

Resources

About