Integration of TomoPy and the ASTRA toolbox for advanced processing and reconstruction of tomographic synchrotron data

Pelt, Daniël M.; Gürsoy, Doğa; Palenstijn, Willem Jan; Sijbers, Jan; Carlo, Francesco De; Batenburg, Kees Joost

doi:10.1107/s1600577516005658

Cited by 109 publications

(69 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The data were acquired with a transmission X-ray microscope (TXM) setup at the 32-ID beamline of Argonne National Laboratory’s Advanced Photon Source [35]. After acquisition, the projections were processed with TomoPy [32] and reconstructed with either TomoPy (all gridrec reconstructions) or the ASTRA toolbox [34] (all SIRT-FBP and SIRT reconstructions), using the recent integration of both toolboxes [18]. …”

Section: Resultsmentioning

confidence: 99%

“…For example, tomographic operations are well suited for graphic processing units (GPUs) [15], enabling significant time reduction by exploiting them in iterative methods [16, 17]. Even when using GPUs, however, it takes hours to compute full three-dimensional iterative reconstructions for typical data sizes in modern large-scale tomographic experiments [18]. A different approach is to use massively parallelized methods on large supercomputers to improve computation time.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Improved tomographic reconstruction of large-scale real-world data by filter optimization

Pelt

Andrade

2016

Adv Struct Chem Imag

Self Cite

View full text Add to dashboard Cite

In advanced tomographic experiments, large detector sizes and large numbers of acquired datasets can make it difficult to process the data in a reasonable time. At the same time, the acquired projections are often limited in some way, for example having a low number of projections or a low signal-to-noise ratio. Direct analytical reconstruction methods are able to produce reconstructions in very little time, even for large-scale data, but the quality of these reconstructions can be insufficient for further analysis in cases with limited data. Iterative reconstruction methods typically produce more accurate reconstructions, but take significantly more time to compute, which limits their usefulness in practice. In this paper, we present the application of the SIRT-FBP method to large-scale real-world tomographic data. The SIRT-FBP method is able to accurately approximate the simultaneous iterative reconstruction technique (SIRT) method by the computationally efficient filtered backprojection (FBP) method, using precomputed experiment-specific filters. We specifically focus on the many implementation details that are important for application on large-scale real-world data, and give solutions to common problems that occur with experimental data. We show that SIRT-FBP filters can be computed in reasonable time, even for large problem sizes, and that precomputed filters can be reused for future experiments. Reconstruction results are given for three different experiments, and are compared with results of popular existing methods. The results show that the SIRT-FBP method is able to accurately approximate iterative reconstructions of experimental data. Furthermore, they show that, in practice, the SIRT-FBP method can produce more accurate reconstructions than standard direct analytical reconstructions with popular filters, without increasing the required computation time.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Improved tomographic reconstruction of large-scale real-world data by filter optimization

Pelt

Andrade

2016

Adv Struct Chem Imag

Self Cite

View full text Add to dashboard Cite

show abstract

“…It is, however, a very topic-specific framework focused on tomography. This limits its possible applications, as it often requires a sizeable effort to extend it to include other data reduction processes such as X-ray fluorescence tomography (Hong et al, 2015;Gursoy et al, 2015) or even other tomography toolboxes (Pelt et al, 2016). As it is highly optimized for use on particular cluster architecture it is hard to roll out a performant framework to different facilities.…”

Section: Big Data Processing: a Solution For Modern Scientific Data Pmentioning

confidence: 99%

Automatic processing of multimodal tomography datasets

Parsons

Price

Wadeson

et al. 2017

J Synchrotron Radiat

View full text Add to dashboard Cite

With the development of fourth-generation high-brightness synchrotrons on the horizon, the already large volume of data that will be collected on imaging and mapping beamlines is set to increase by orders of magnitude. As such, an easy and accessible way of dealing with such large datasets as quickly as possible is required in order to be able to address the core scientific problems during the experimental data collection. Savu is an accessible and flexible big data processing framework that is able to deal with both the variety and the volume of data of multimodal and multidimensional scientific datasets output such as those from chemical tomography experiments on the I18 microfocus scanning beamline at Diamond Light Source.

show abstract

“…Tomography is a widely used imaging technique at synchrotron light sources (Gü rsoy et al, 2015a,b;Mohan et al, 2015;Qi & Leahy, 2006;Duke et al, 2015;Chen et al, 2012;Pelt et al, 2016). Iterative tomographic reconstruction typically provides higher-quality images than the filtered-back projection method does, because of the better reduction in noise and in missing wedge artifacts.…”

Section: Related Workmentioning

confidence: 99%

Optimization of tomographic reconstruction workflows on geographically distributed resources

Biçer

Gürsoy

Kettimuthu

et al. 2016

J Synchrotron Radiat

Self Cite

View full text Add to dashboard Cite

New technological advancements in synchrotron light sources enable data acquisitions at unprecedented levels. This emergent trend affects not only the size of the generated data but also the need for larger computational resources. Although beamline scientists and users have access to local computational resources, these are typically limited and can result in extended execution times. Applications that are based on iterative processing as in tomographic reconstruction methods require high-performance compute clusters for timely analysis of data. Here, time-sensitive analysis and processing of Advanced Photon Source data on geographically distributed resources are focused on. Two main challenges are considered: (i) modeling of the performance of tomographic reconstruction workflows and (ii) transparent execution of these workflows on distributed resources. For the former, three main stages are considered: (i) data transfer between storage and computational resources, (i) wait/queue time of reconstruction jobs at compute resources, and (iii) computation of reconstruction tasks. These performance models allow evaluation and estimation of the execution time of any given iterative tomographic reconstruction workflow that runs on geographically distributed resources. For the latter challenge, a workflow management system is built, which can automate the execution of workflows and minimize the user interaction with the underlying infrastructure. The system utilizes Globus to perform secure and efficient data transfer operations. The proposed models and the workflow management system are evaluated by using three high-performance computing and two storage resources, all of which are geographically distributed. Workflows were created with different computational requirements using two compute-intensive tomographic reconstruction algorithms. Experimental evaluation shows that the proposed models and system can be used for selecting the optimum resources, which in turn can provide up to 3.13Â speedup (on experimented resources). Moreover, the error rates of the models range between 2.1 and 23.3% (considering workflow execution times), where the accuracy of the model estimations increases with higher computational demands in reconstruction tasks.

show abstract

Integration of TomoPy and the ASTRA toolbox for advanced processing and reconstruction of tomographic synchrotron data

Abstract: The integration of two Python toolboxes used for processing tomographic data, TomoPy and the ASTRA toolbox, is presented.

Cited by 109 publications

References 33 publications

Improved tomographic reconstruction of large-scale real-world data by filter optimization

Improved tomographic reconstruction of large-scale real-world data by filter optimization

Automatic processing of multimodal tomography datasets

Optimization of tomographic reconstruction workflows on geographically distributed resources

Contact Info

Product

Resources

About