One of the fundamental challenges in supervised learning for multimodal image registration is the lack of ground-truth for voxel-level spatial correspondence. This work describes a method to infer voxel-level transformation from higher-level correspondence information contained in anatomical labels. We argue that such labels are more reliable and practical to obtain for reference sets of image pairs than voxel-level correspondence. Typical anatomical labels of interest may include solid organs, vessels, ducts, structure boundaries and other subject-specific ad hoc landmarks. The proposed end-to-end convolutional neural network approach aims to predict displacement fields to align multiple labelled corresponding structures for individual image pairs during the training, while only unlabelled image pairs are used as the network input for inference. We highlight the versatility of the proposed strategy, for training, utilising diverse types of anatomical labels, which need not to be identifiable over all training image pairs. At inference, the resulting 3D deformable image registration algorithm runs in real-time and is fully-automated without requiring any anatomical labels or initialisation. Several network architecture variants are compared for registering T2-weighted magnetic resonance images and 3D transrectal ultrasound images from prostate cancer patients. A median target registration error of 3.6 mm on landmark centroids and a median Dice of 0.87 on prostate glands are achieved from cross-validation experiments, in which 108 pairs of multimodal images from 76 patients were tested with high-quality anatomical labels.

Spatially aligning medical images from different modalities remains a challenging task, especially for intraoperative applications that require fast and robust algorithms. We propose a weakly-supervised, label-driven formulation for learning 3D voxel correspondence from higher-level label correspondence, thereby bypassing classical intensity-based image similarity measures. During training, a convolutional neural network is optimised by outputting a dense displacement field (DDF) that warps a set of available anatomical labels from the moving image to match their corresponding counterparts in the fixed image. These label pairs, including solid organs, ducts, vessels, point landmarks and other ad hoc structures, are only required at training time and can be spatially aligned by minimising a crossentropy function of the warped moving label and the fixed label. During inference, the trained network takes a new image pair to predict an optimal DDF, resulting in a fullyautomatic, label-free, real-time and deformable registration. For interventional applications where large global transformation prevails, we also propose a neural network architecture to jointly optimise the global-and local displacements. Experiment results are presented based on cross-validating registrations of 111 pairs of T2-weighted magnetic resonance images and 3D transrectal ultrasound images from prostate cancer patients with a total of over 4000 anatomical labels, yielding a median target registration error of 4.2 mm on landmark centroids and a median Dice of 0.88 on prostate glands.Index Terms-multimodal medical image registration, weakly-supervised learning, prostate cancer

A novel technique for imaging based on ultrawideband (UWB) microwave signals is introduced. Specifically, the procedure deals with the Huygens Principle (HP); using HP to forward-propagate the waves removes the need to solve inverse problems and, consequently, no matrix generation/inversion is required. Together with its simplicity, the methodology permits the capture of contrast-the extent to which different media can be discriminated in the final image. Moreover, UWB allows all the information in the frequency domain to be utilized by combining the information from the individual frequencies to construct a consistent image. It follows that the methodology can identify the presence and location of significant scatterers inside a volume. Validation of the technique through both simulations and measurements on cylinder and spheres with inclusions has been performed. Application of the proposed technique to medical imaging is envisaged

We present an initial experimental validation of a microwave tomography (MWT) prototype for brain stroke detection and classification using the distorted Born iterative method, two-step iterative shrinkage thresholding (DBIM-TwIST) algorithm. The validation study consists of first preparing and characterizing gel phantoms which mimic the structure and the dielectric properties of a simplified brain model with a haemorrhagic or ischemic stroke target. Then, we measure the S-parameters of the phantoms in our experimental prototype and process the scattered signals from 0.5 to 2.5 GHz using the DBIM-TwIST algorithm to estimate the dielectric properties of the reconstruction domain. Our results demonstrate that we are able to detect the stroke target in scenarios where the initial guess of the inverse problem is only an approximation of the true experimental phantom. Moreover, the prototype can differentiate between haemorrhagic and ischemic strokes based on the estimation of their dielectric properties.

We describe an adversarial learning approach to constrain convolutional neural network training for image registration, replacing heuristic smoothness measures of displacement fields often used in these tasks. Using minimallyinvasive prostate cancer intervention as an example application, we demonstrate the feasibility of utilizing biomechanical simulations to regularize a weakly-supervised anatomical-label-driven registration network for aligning pre-procedural magnetic resonance (MR) and 3D intra-procedural transrectal ultrasound (TRUS) images. A discriminator network is optimized to distinguish the registration-predicted displacement fields from the motion data simulated by finite element analysis. During training, the registration network simultaneously aims to maximize similarity between anatomical labels that drives image alignment and to minimize an adversarial generator loss that measures divergence between the predicted-and simulated deformation. The end-to-end trained network enables efficient and fully-automated registration that only requires an MR and TRUS image pair as input, without anatomical labels or simulated data during inference. 108 pairs of labelled MR and TRUS images from 76 prostate cancer patients and 71,500 nonlinear finite-element simulations from 143 different patients were used for this study. We show that, with only gland segmentation as training labels, the proposed method can help predict physically plausible deformation without any other smoothness penalty. Based on cross-validation experiments using 834 pairs of independent validation landmarks, the proposed adversarial-regularized registration achieved a target registration error of 6.3 mm that is significantly lower than those from several other regularization methods.

This paper presents preliminary results of an innovative microwave imaging apparatus for breast lesions detection. Specifically, a Huygens Principle based method is employed to process the microwave signals and to build the respective microwave images. The apparatus has been first tested on phantoms. Next, its performance has been verified through clinical examinations on 22 healthy breasts and on 29 breast having lesions, using as gold standard the output of the radiologist study review obtained using conventional techniques. Specifically, we introduce a metric, which is the ratio between maximum and average of the image intensity (MAX/AVG). We found that MAX/AVG of microwave images can be used for classifying breasts containing lesions. In addition, using MAX/AVG as classification parameter, receiver operating characteristic curves have been empirically determined. Furthermore, for one randomly selected breast having lesion, we have demonstrated that the localization of the inclusion acquired through microwave imaging is compatible with mammography images.

Deep-learning-based segmentation tools are yielding higher reported segmentation accuracies for many medical image segmentation problems. However, inter-site variability in medical image acquisition protocols and quality can challenge the translation of these tools to data from unseen sites. This study quantifies the impact of inter-site variability on the accuracy of deep-learning-based segmentation of the prostate from magnetic resonance (MR) images, and evaluates two strategies for mitigating the performance discrepancies for data from unseen sites: training on multi-site datasets and retraining with data from the unseen site. Using 424 T2-weighted prostate MR images from six sites, we compare the segmentation accuracy of three deep-learning-based networks trained on data from a single site and on various configurations of data from multiple sites. We found that the segmentation accuracy of a single-site network was substantially worse on data from unseen sites than on data from the training site, and that training on multi-site data gives only marginal improvement. However, including as few as 8 subjects from the unseen site, e.g. during commissioning each new clinical system, yields substantial improvement (regaining 75% of the difference).

Microwave imaging has received increasing attention in the last decades, motivated by its application in diagnostic imaging. Such effort has been encouraged by the fact that, at microwave frequencies, it is possible to distinguish between tissues with different dielectric properties. In such framework, a novel microwave device is presented here. The apparatus, consisting of two antennas operating in air, is completely safe and non-invasive since it does not emit any ionizing radiation and it can be used for breast lesion detection without requiring any breast crushing. We use Huygens Principle to provide a novel understanding into microwave imaging; specifically, the algorithm based on this principle provides images which represent homogeneity maps of the dielectric properties (dielectric constant and/or conductivity). The experimental results on phantoms having inclusions with different dielectric constants are presented here. In addition, the capability of the device to detect breast lesions has been verified through clinical examinations on 51 breasts.We introduce a metric to measure the non-homogenous behaviour of the image, establishing a modality to detect the presence of inclusions inside phantoms and, similarly, the presence of a lesion inside a breast.