Task adapted reconstruction for inverse problems

Abstract

The paper considers the problem of performing a post-processing task defined on a model parameter that is only observed indirectly through noisy data in an ill-posed inverse problem. A key aspect is to formalize the steps of reconstruction and post-processing as appropriate estimators (non-randomized decision rules) in statistical estimation problems. The implementation makes use of (deep) neural networks to provide a differentiable parametrization of the family of estimators for both steps. These networks are combined and jointly trained against suitable supervised training data in order to minimize a joint differentiable loss function, resulting in an end-to-end task adapted reconstruction method. The suggested framework is generic, yet adaptable, with a plug-and-play structure for adjusting both the inverse problem and the post-processing task at hand. More precisely, the data model (forward operator and statistical model of the noise) associated with the inverse problem is exchangeable, e.g., by using neural network architecture given by a learned iterative method. Furthermore, any post-processing that can be encoded as a trainable neural network can be used. The approach is demonstrated on joint tomographic image reconstruction, classification and joint tomographic image reconstruction segmentation.

A Note on Machine Learning Approach for Computational Imaging

“…A proof-of-concept study regarding this was given by [80], where we showed that an image abnormality detection task may induce a significantly different image quality metric from pixel-level metric. Works in computer vision [42,41] and radiology [34] also suggest potentials of task-driven imaging, and a theoretical framework was introduced by [2]. Therefore, the unification may lead to more effective and practically relevant ways to evaluating imaging qualities and more economic sensing strategies for a given task or a set of tasks.…”

Deep Microlocal Reconstruction for Limited-Angle Tomography

“…This propagated wavefront set, along with the characterisation of the visible wavefront set, is used for setting up the DNN for wavefront set inpainting, which recovers the invisible part of the wavefront set of a signal from the visible part. This DNN for wavefront set inpainting is then combined with the Learned Primal-Dual network for reconstruction, and both DNNs are trained jointly following the task-adapted reconstruction paradigm outlined in [1], see Section 5.6 for further details.…”

“…where C ∈ (0, 1] and 1 , 2 are appropriate distance measures on discretised images and discretised wavefront sets that will be discussed in Subsection 3.4 below. This strategy that jointly trains a reconstruction and a task (wavefront set inpainting) falls in the framework of task-adapted reconstruction was introduced in [1].…”

Reconstruction and segmentation from sparse sequential X-ray measurements of wood logs

“…For instance, in medical imaging the CT image might be used for identification and quantification of cancer tissue [11]. Thus, in recent years some researchers have also increasingly considered to combine both tasks in a joint framework [12], [13], [14]. Nevertheless, we will concentrate here on a separated approach, but keep the segmentation quality in mind as evaluation criterion of reconstruction quality rather than quantitative reconstruction errors.…”