This paper tries to give a gentle introduction to deep learning in medical image processing, proceeding from theoretical foundations to applications. We first discuss general reasons for the popularity of deep learning, including several major breakthroughs in computer science. Next, we start reviewing the fundamental basics of the perceptron and neural networks, along with some fundamental theory that is often omitted. Doing so allows us to understand the reasons for the rise of deep learning in many application domains. Obviously medical image processing is one of these areas which has been largely affected by this rapid progress, in particular in image detection and recognition, image segmentation, image registration, and computer-aided diagnosis. There are also recent trends in physical simulation, modelling, and reconstruction that have led to astonishing results. Yet, some of these approaches neglect prior knowledge and hence bear the risk of producing implausible results. These apparent weaknesses highlight current limitations of deep learning. However, we also briefly discuss promising approaches that might be able to resolve these problems in the future.

An abdominal aortic aneurysm (AAA) is a balloon-like dilation of the aorta, which is potentially fatal in case of rupture. Computational finite element (FE) analysis is a promising approach to a more accurate and patient-specific rupture risk prediction. AAA wall strength and rupture potential index (RPI) calculation are implemented in our FE software. Static structural FE simulations are performed on n = 30 non-ruptured asymptomatic, n = 9 non-ruptured symptomatic, and n = 14 ruptured AAAs. We calculate maximum values for diameter, wall displacement, strain, stress, and RPI as well as minimum wall strength for every AAA. All investigated quantities, except minimum strength, show statistically significant differences between non-ruptured asymptomatic and symptomatic/ruptured AAAs. Maximum wall stress and especially the RPI are notably increased for symptomatic and ruptured AAAs. The biggest difference is found to be the RPI (Δ = 44.9%, p = 8.0e-5). Lowest RPI obtained for symptomatic or ruptured AAAs is 0.3. The RPI of more than 55% of the investigated asymptomatic AAAs falls below this value. Maximum wall stress and maximum RPI criteria enable a reliable rupture risk evaluation for AAAs. Especially in the diameter range where surgical indication is not obvious, the RPI holds great potential for improvement of clinical decisions.

Electroluminescence (EL) imaging is a useful modality for the inspection of photovoltaic (PV) modules. EL images provide high spatial resolution, which makes it possible to detect even finest defects on the surface of PV modules. However, the analysis of EL images is typically a manual process that is expensive, time-consuming, and requires expert knowledge of many different types of defects.In this work, we investigate two approaches for automatic detection of such defects in a single image of a PV cell. The approaches differ in their hardware requirements, which are dictated by their respective application scenarios. The more hardware-efficient approach is based on hand-crafted features that are classified in a Support Vector Machine (SVM). To obtain a strong performance, we investigate and compare various processing variants. The more hardware-demanding approach uses an end-to-end deep Convolutional Neural Network (CNN) that runs on a Graphics Processing Unit (GPU). Both approaches are trained on 1,968 cells extracted from high resolution EL intensity images of mono-and polycrystalline PV modules. The CNN is more accurate, and reaches an average accuracy of 88.42 %. The SVM achieves a slightly lower average accuracy of 82.44 %, but can run on arbitrary hardware. Both automated approaches make continuous, highly accurate monitoring of PV cells feasible.

Robust and fast detection of anatomical structures is a prerequisite for medical image analysis. Current solutions for anatomy detection are typically based on machine learning and are subject to several limitations, including the use of suboptimal feature engineering techniques and most importantly the use of computationally suboptimal search-schemes. To address these issues, we propose a method that follows a new paradigm by reformulating the detection problem as a behavior learning task for an artificial agent. We couple the modeling of the anatomy appearance and the object search in a unified behavioral framework, using the capabilities of deep reinforcement learning and multi-scale image analysis. In other words, an artificial agent is trained not only to distinguish the target anatomical object from the rest of the body but also how to find the object by learning and following an optimal navigation path to the target object in the imaged volumetric space. We evaluate our approach on 1487 3D-CT volumes from 532 patients and show that we significantly outperform state-of-the-art solutions on detecting several anatomical structures with no failed cases, while also improving the detection accuracy by 20-30%. Most importantly, we improve the detection-speed of the reference methods by 2-3 orders of magnitude, achieving unmatched real-time performance on large 3D-CT scans.

In this paper, we present a new deep learning framework for 3-D tomographic reconstruction. To this end, we map filtered back-projection-type algorithms to neural networks. However, the back-projection cannot be implemented as a fully connected layer due to its memory requirements. To overcome this problem, we propose a new type of cone-beam back-projection layer, efficiently calculating the forward pass. We derive this layer's backward pass as a projection operation. Unlike most deep learning approaches for reconstruction, our new layer permits joint optimization of correction steps in volume and projection domain. Evaluation is performed numerically on a public data set in a limited angle setting showing a consistent improvement over analytical algorithms while keeping the same computational test-time complexity by design. In the region of interest, the peak signal-to-noise ratio has increased by 23%. In addition, we show that the learned algorithm can be interpreted using known concepts from cone beam reconstruction: the network is able to automatically learn strategies such as compensation weights and apodization windows.

Both the clinically established diameter criterion and novel approaches of computational finite element (FE) analyses for rupture risk stratification of abdominal aortic aneurysms (AAA) are based on assumptions of population-averaged, uniform material properties for the AAA wall. The presence of inter-patient and intra-patient variations in material properties is known, but has so far not been addressed sufficiently. In order to enable the preoperative estimation of patient-specific AAA wall properties in the future, we investigated the relationship between non-invasively assessable clinical parameters and experimentally measured AAA wall properties. We harvested n = 163 AAA wall specimens (n = 50 patients) during open surgery and recorded the exact excision sites. Specimens were tested for their thickness, elastic properties, and failure loads using uniaxial tensile tests. In addition, 43 non-invasively assessable patient-specific or specimen-specific parameters were obtained from recordings made during surgery and patient charts. Experimental results were correlated with the non-invasively assessable parameters and simple regression models were created to mathematically describe the relationships. Wall thickness was most significantly correlated with the metabolic activity at the excision site assessed by PET/CT (ρ = 0.499, P = 4 × 10(-7)) and to thrombocyte counts from laboratory blood analyses (ρ = 0.445, P = 3 × 10(-9)). Wall thickness was increased in patients suffering from diabetes mellitus, while it was significantly thinner in patients suffering from chronic kidney disease (CKD). Elastic AAA wall properties had significant correlations with the metabolic activity at the excision site (PET/CT), with existent calcifications, and with the diameter of the non-dilated aorta proximal to the AAA. Failure properties (wall strength and failure tension) had correlations with the patient's medical history and with results from laboratory blood analyses. Interestingly, AAA wall failure tension was significantly reduced for patients with CKD and elevated blood levels of potassium and urea, respectively, both of which are associated with kidney disease. This study is a first step to a future preoperative estimation of AAA wall properties. Results can be conveyed to both the diameter criterion and FE analyses to refine rupture risk prediction. The fact that AAA wall from patients suffering from CKD featured reduced failure tension implies an increased AAA rupture risk for this patient group at comparably smaller AAA diameters.

We describe an approach for incorporating prior knowledge into machine learning algorithms. We aim at applications in physics and signal processing in which we know that certain operations must be embedded into the algorithm. Any operation that allows computation of a gradient or sub-gradient towards its inputs is suited for our framework. We derive a maximal error bound for deep nets that demonstrates that inclusion of prior knowledge results in its reduction. Furthermore, we also show experimentally that known operators reduce the number of free parameters. We apply this approach to various tasks ranging from CT image reconstruction over vessel segmentation to the derivation of previously unknown imaging algorithms. As such the concept is widely applicable for many researchers in physics, imaging, and signal processing. We assume that our analysis will support further investigation of known operators in other fields of physics, imaging, and signal processing.

Large bioacoustic archives of wild animals are an important source to identify reappearing communication patterns, which can then be related to recurring behavioral patterns to advance the current understanding of intra-specific communication of non-human animals. A main challenge remains that most large-scale bioacoustic archives contain only a small percentage of animal vocalizations and a large amount of environmental noise, which makes it extremely difficult to manually retrieve sufficient vocalizations for further analysis – particularly important for species with advanced social systems and complex vocalizations. In this study deep neural networks were trained on 11,509 killer whale ( Orcinus orca ) signals and 34,848 noise segments. The resulting toolkit ORCA-SPOT was tested on a large-scale bioacoustic repository – the Orchive – comprising roughly 19,000 hours of killer whale underwater recordings. An automated segmentation of the entire Orchive recordings (about 2.2 years) took approximately 8 days. It achieved a time-based precision or positive-predictive-value (PPV) of 93.2% and an area-under-the-curve (AUC) of 0.9523. This approach enables an automated annotation procedure of large bioacoustics databases to extract killer whale sounds, which are essential for subsequent identification of significant communication patterns. The code will be publicly available in October 2019 to support the application of deep learning to bioaoucstic research. ORCA-SPOT can be adapted to other animal species.