Data reduction for serial crystallography using a robust peak finder

Abstract

A peak-finding algorithm for serial crystallography (SX) data analysis based on the principle of `robust statistics' has been developed. Methods which are statistically robust are generally more insensitive to any departures from model assumptions and are particularly effective when analysing mixtures of probability distributions. For example, these methods enable the discretization of data into a group comprising inliers (i.e. the background noise) and another group comprising outliers (i.e. Bragg peaks). Our robust statistics algorithm has two key advantages, which are demonstrated through testing using multiple SX data sets. First, it is relatively insensitive to the exact value of the input parameters and hence requires minimal optimization. This is critical for the algorithm to be able to run unsupervised, allowing for automated selection or `vetoing' of SX diffraction data. Secondly, the processing of individual diffraction patterns can be easily parallelized. This means that it can analyse data from multiple detector modules simultaneously, making it ideally suited to real-time data processing. These characteristics mean that the robust peak finder (RPF) algorithm will be particularly beneficial for the new class of MHz X-ray free-electron laser sources, which generate large amounts of data in a short period of time.

Data reduction for X-ray serial crystallography using machine learning

“…Considering the nature of SFX experiments, it is clear that only crystal hits with Bragg peaks are useful for downstream analysis (Wiedorn et al, 2018). Therefore, existing statistical methods utilize peak finding to identify and discern diffraction patterns that contain Bragg peaks and remove any patterns which only contain empty shots, resulting in considerable data reduction (Hadian-Jazi et al, 2021Barty et al, 2014;Mariani et al, 2016;Thayer et al, 2017). Typically, peak finding methods attempt to find all the Bragg peaks in these diffraction patterns, which can be computationally expensive.…”

Automatic bad-pixel mask maker for X-ray pixel detectors with application to serial crystallography

“…If not treated correctly, bad pixels that are too bright compared with their surroundings will be identified as Bragg peaks. Also, bad pixels with intensities that are too low can bias the estimated model parameters for Bragg peaks, depending on the robustness of the Bragg peak finder (Hadian-Jazi et al, 2021). We compared the results of data analysis using the published masks available in the data sets with results using masks generated by RMM.…”

“…In this appendix we discuss how changing the parameter affects the results of the analysis pipeline described in Section 4.1, using the RPF software (Hadian-Jazi et al, 2021).…”

“…The data set is discussed in more detail in Section 4.1. A state-of-the-art Bragg peak finder called RPF (Hadian-Jazi et al, 2021) was used to detect Bragg peaks in these diffraction patterns. Fig.…”

Crystallite size distribution by two-dimensional XRD

“…There are many software packages available to identify the peaks/spots. For example, many single-crystal diffraction software packages have peak finding routines to evaluate the spots in 2D diffraction patterns (Schlichting, 2015; Hadian-Jazi et al, 2021). Peak/spot finding may involve user-selected threshold for determining a peak above the background or separation of merging spots.…”