Calibration by differentiation – Self‐supervised calibration for X‐ray microscopy using a differentiable cone‐beam reconstruction operator

Abstract

High‐resolution X‐ray microscopy (XRM) is gaining interest for biological investigations of extremely small‐scale structures. XRM imaging of bones in living mice could provide new insights into the emergence and treatment of osteoporosis by observing osteocyte lacunae, which are holes in the bone of few micrometres in size. Imaging living animals at that resolution, however, is extremely challenging and requires very sophisticated data processing converting the raw XRM detector output into reconstructed images. This paper presents an open‐source, differentiable reconstruction pipeline for XRM data which analytically computes the final image from the raw measurements. In contrast to most proprietary reconstruction software, it offers the user full control over each processing step and, additionally, makes the entire pipeline deep learning compatible by ensuring differentiability. This allows fitting trainable modules both before and after the actual reconstruction step in a purely data‐driven way using the gradient‐based optimizers of common deep learning frameworks. The value of such differentiability is demonstrated by calibrating the parameters of a simple cupping correction module operating on the raw projection images using only a self‐supervisory quality metric based on the reconstructed volume and no further calibration measurements. The retrospective calibration directly improves image quality as it avoids cupping artefacts and decreases the difference in grey values between outer and inner bone by 68–94%. Furthermore, it makes the reconstruction process entirely independent of the XRM manufacturer and paves the way to explore modern deep learning reconstruction methods for arbitrary XRM and, potentially, other flat‐panel computed tomography systems. This exemplifies how differentiable reconstruction can be leveraged in the context of XRM and, hence, is an important step towards the goal of reducing the resolution limit of in vivo bone imaging to the single micrometre domain.

Trainable joint bilateral filters for enhanced prediction stability in low-dose CT

“…Previously, Maier et al proved that including physical knowledge in terms of known operators in neural networks reduces the absolute error bound of the model [20][21][22] . Consequently, different image processing pipelines were proposed, employing physical assumptions about noise characteristics to leverage prediction reliability of DL-based methods in the context of image denoising 23,24 .…”

“…Therefore, deep learning (DL)-based denoising methods gained interest due to their flexibility, strong performance, and data-driven optimization [12][13][14][15][16][17] . However, deep neural networks usually do not robustly generalize beyond their finite training data distribution, which so far limits clinical applications of DL-based denoising for low-dose CT 18,19 .Previously, Maier et al proved that including physical knowledge in terms of known operators in neural networks reduces the absolute error bound of the model [20][21][22] . Consequently, different image processing pipelines were proposed, employing physical assumptions about noise characteristics to leverage prediction reliability of DL-based methods in the context of image denoising 23,24 .…”