Scientific Image Restoration Anywhere

Abeykoon, Vibhatha; Liu, Zhengchun; Kettimuthu, Rajkumar; Fox, Geoffrey; Foster, Ian

doi:10.1109/xloop49562.2019.00007

Cited by 15 publications

(9 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While the brightness increases with these new accelerator upgrades will be significant, these new sources will not reach their full potential if limited to using detectors of the type available today, or those presently seen on the horizon. In addition, application scientists have increasingly turned to AI to analyze data [4,5,6,7,8]. AI-accelerated workflows have been shown not only to be fast enough to keep up with experiments, but also to overcome experimental restrictions of conventional methods.…”

Section: Timeliness or Maturitymentioning

confidence: 99%

Pushing compute and AI onto detector silicon

Miceli¹,

Yoshii²,

Foster³

2022

Preprint

View full text Add to dashboard Cite

Pixel detectors are at the heart of advances in imaging. Astronomy has been revolutionized by the use of highthroughput detectors for surveying supernovas, shedding light on expansion rates of the universe, and exoplanet searches using the wobble of faint stars. Biochemistry has been transformed first by developing megapixel detectors for X-ray crystallography, and more recently, by using high frame rate detectors to correct for drift and enable atomic resolution structure determination in cryo-electron microscopy. Materials scientists are gaining new insights into subtle electron bonding arrangements using high dynamic range electron detectors as part of phasing coherent diffraction patterns. Beyond scientific advances, the photography industry has been revolutionized by the transition to electronic image detectors.However, in order to take full advantage of the DOE's billion-dollar investments into the next-generation research infrastructure (e.g., exascale, light sources, colliders), advances are required not only in detector technology but also in computing and specifically AI. Let us consider an example from X-ray science. Nanoscale X-ray imaging is a crucial tool to enable a wide range of scientific explorations from materials science and biology to mechanical and civil engineering. The next-generation light sources will increase the X-ray beam brightness and coherent flux by 100 to 1,000 times. In order to image larger samples, the continuous frame rate of pixel array detectors must be increased, approaching 1 MHz, which requires several Tbps (aggregated) to transfer pixel data out to a data acquisition system. Using 65-nm CMOS technology, an optimistic raw data rate off such a chip is about 100-200 Gbps. However, a continuous 1 MHz detector with only 256 × 256 pixels at 16-bit resolution, for example, will require 1,000 Gbps (i.e., 1 Tbps) bandwidth off the chip! It is impractical to have multiple high-speed transceivers running in parallel to provide such bandwidth and represents the first data bottleneck. New approaches are necessary to reduce the data size by performing data compression or AI-based feature extraction directly inside a detector silicon chip in a streaming manner before sending it off-chip.The dividing line between edge and exascale computing is often ambiguous. However for scientific imaging detectors, the ultimate location to incorporate edge compute is directly on the front-end detector silicon. Many of these scientific detectors are constructed from application-specific integrated circuits (ASICs). Until recently, pixel detector ASICs have been used mainly for analog signal processing of the charge from the sensor layer and the transmission of raw pixel data off the detector ASIC. However, with the availability of more advanced ASIC technology nodes for scientific application, more digital functionality from the computing domains (e.g., compression and feature extraction) can be integrated directly into the detector ASIC (e.g., on the chip's periphery or edge) to overcome off-chip ba...

show abstract

Section: Timeliness or Maturitymentioning

confidence: 99%

Pushing compute and AI onto detector silicon

Miceli¹,

Yoshii²,

Foster³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…First, TomoGAN uses a GAN architecture, with adversarial loss, to help train the generator, and a pre-trained VGG [42] network; results presented below suggest that the adversarial loss avoids artifacts. Second, our generator, although also based on U-Net, has three U-Net boxes, as shown in Figure 3 instead of the four in FBPConvNet, and no batch normalization layer [20], two factors that reduce computation and memory needs for inference (e.g., TomoGAN can efficiently run on Google edge TPU [62]). For example, with one NVIDIA Tesla V100 card and a batch size of eight (minimizing DL framework overheads), TomoGAN and FBPConvNet take an average of 30ms and 90ms to process one 1024×1024 image, respectively: TomoGAN is three times faster.…”

Section: Comparison With Other Solutions On Experimental Datasetsmentioning

confidence: 99%

TomoGAN: low-dose synchrotron x-ray tomography with generative adversarial networks: discussion

et al. 2020

Self Cite

View full text Add to dashboard Cite

Synchrotron-based x-ray tomography is a noninvasive imaging technique that allows for reconstructing the internal structure of materials at high spatial resolutions from tens of micrometers to a few nanometers. In order to resolve sample features at smaller length scales, however, a higher radiation dose is required. Therefore, the limitation on the achievable resolution is set primarily by noise at these length scales. We present TomoGAN, a denoising technique based on generative adversarial networks, for improving the quality of reconstructed images for low-dose imaging conditions. We evaluate our approach in two photon-budget-limited experimental conditions: (1) sufficient number of low-dose projections (based on Nyquist sampling), and (2) insufficient or limited number of high-dose projections. In both cases the angular sampling is assumed to be isotropic, and the photon budget throughout the experiment is fixed based on the maximum allowable radiation dose on the sample. Evaluation with both simulated and experimental datasets shows that our approach can significantly reduce noise in reconstructed images, improving the structural similarity score of simulation and experimental data from 0.18 to 0.9 and from 0.18 to 0.41, respectively. Furthermore, the quality of the reconstructed images with filtered back projection followed by our denoising approach exceeds that of reconstructions with the simultaneous iterative reconstruction technique, showing the computational superiority of our approach.

show abstract

“…MemXCT is a highly optimized reconstruction engine for large-scale tomography datasets [10]. In this work, we extended our efficient stream reconstruction data analysis pipeline [8,54,55] with denoising capabilities [6,56].…”

Section: Related Workmentioning

confidence: 99%

Deep Learning Accelerated Light Source Experiments

Liu

Biçer

Kettimuthu

et al. 2019

2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS)

Self Cite

View full text Add to dashboard Cite

Experimental protocols at synchrotron light sources typically process and validate data only after an experiment has completed, which can lead to undetected errors and cannot enable online steering. Real-time data analysis can enable both detection of, and recovery from, errors, and optimization of data acquisition. However, modern scientific instruments, such as detectors at synchrotron light sources, can generate data at GBs/sec rates. Data processing methods such as the widely used computational tomography usually require considerable computational resources, and yield poor quality reconstructions in the early stages of data acquisition when available views are sparse. We describe here how a deep convolutional neural network can be integrated into the real-time streaming tomography pipeline to enable better-quality images in the early stages of data acquisition. Compared with conventional streaming tomography processing, our method can significantly improve tomography image quality, deliver comparable images using only 32% of the data needed for conventional streaming processing, and save 68% experiment time for data acquisition.

show abstract

Scientific Image Restoration Anywhere

Cited by 15 publications

References 32 publications

Pushing compute and AI onto detector silicon

Pushing compute and AI onto detector silicon

TomoGAN: low-dose synchrotron x-ray tomography with generative adversarial networks: discussion

Deep Learning Accelerated Light Source Experiments

Contact Info

Product

Resources

About