In this paper, we present a Convolutional Neural Network (CNN) regression approach to address the two major limitations of existing intensity-based 2-D/3-D registration technology: 1) slow computation and 2) small capture range. Different from optimization-based methods, which iteratively optimize the transformation parameters over a scalar-valued metric function representing the quality of the registration, the proposed method exploits the information embedded in the appearances of the digitally reconstructed radiograph and X-ray images, and employs CNN regressors to directly estimate the transformation parameters. An automatic feature extraction step is introduced to calculate 3-D pose-indexed features that are sensitive to the variables to be regressed while robust to other factors. The CNN regressors are then trained for local zones and applied in a hierarchical manner to break down the complex regression task into multiple simpler sub-tasks that can be learned separately. Weight sharing is furthermore employed in the CNN regression model to reduce the memory footprint. The proposed approach has been quantitatively evaluated on 3 potential clinical applications, demonstrating its significant advantage in providing highly accurate real-time 2-D/3-D registration with a significantly enlarged capture range when compared to intensity-based methods.

Automatic parsing of anatomical objects in X-ray images is critical to many clinical applications in particular towards image-guided invention and workflow automation. Existing deep network models require a large amount of labeled data. However, obtaining accurate pixelwise labeling in X-ray images relies heavily on skilled clinicians due to the large overlaps of anatomy and the complex texture patterns. On the other hand, organs in 3D CT scans preserve clearer structures as well as sharper boundaries and thus can be easily delineated. In this paper, we propose a novel model framework for learning automatic X-ray image parsing from labeled CT scans. Specifically, a Dense Image-to-Image network (DI2I) for multi-organ segmentation is first trained on Xray like Digitally Reconstructed Radiographs (DRRs) rendered from 3D CT volumes. Then we introduce a Task Driven Generative Adversarial Network (TD-GAN) architecture to achieve simultaneous style transfer and parsing for unseen real X-ray images. TD-GAN consists of a modified cycle-GAN substructure for pixel-to-pixel translation between DRRs and X-ray images and an added module leveraging the pre-trained DI2I to enforce segmentation consistency. The TD-GAN framework is general and can be easily adapted to other learning tasks. In the numerical experiments, we validate the proposed model on 815 DRRs and 153 topograms. While the vanilla DI2I without any adaptation fails completely on segmenting the topograms, the proposed model does not require any topogram labels and is able to provide a promising average dice of 85% which achieves the same level accuracy of supervised training (88%).Disclaimer: This feature is based on research, and is not commercially available. Due to regulatory reasons its future availability cannot be guaranteed.

Robust image registration in medical imaging is essential for comparison or fusion of images, acquired from various perspectives, modalities or at different times. Typically, an objective function needs to be minimized assuming specific a priori deformation models and predefined or learned similarity measures. However, these approaches have difficulties to cope with large deformations or a large variability in appearance. Using modern deep learning (DL) methods with automated feature design, these limitations could be resolved by learning the intrinsic mapping solely from experience. We investigate in this paper how DL could help organ-specific (ROI-specific) deformable registration, to solve motion compensation or atlas-based segmentation problems for instance in prostate diagnosis. An artificial agent is trained to solve the task of non-rigid registration by exploring the parametric space of a statistical deformation model built from training data. Since it is difficult to extract trustworthy ground-truth deformation fields, we present a training scheme with a large number of synthetically deformed image pairs requiring only a small number of real inter-subject pairs. Our approach was tested on inter-subject registration of prostate MR data and reached a median DICE score of .88 in 2-D and .76 in 3-D, therefore showing improved results compared to state-of-the-art registration algorithms.

BackgroundHepatocellular carcinoma (HCC) is a typical malignancy in a background of chronic inflammation. Th17 cells (a major source of IL-17) constitute crucial components of infiltrating inflammatory/immune cells in HCC and can amplify inflammatory response via binding to interleukin-17 receptor (IL-17R). Thus, we investigated the expression and clinical significance of IL-17 and IL-17 receptor family cytokines in HCC.MethodsThe expression and prognostic value of IL-17 and IL-17R (A-E) were examined in 300 HCC patients after resection. Six Th17 associated cytokines in serum (n = 111) were quantified using enzyme-linked immunosorbent assays. Phenotypic features of IL-17+ CD4+ T cells were determined by flow cytometry analysis.ResultsHigh expression of intratumoral IL-17 and IL1-7RE were significantly associated with poorer survival (p = 0.016 and <0.001, respectively) and increased recurrence (both P < 0.001) of HCC patients. Moreover, intratumoral IL-17, individually or synergistically with IL-17RE, could predict HCC early recurrence and late recurrence. Also, peritumoral IL-17RE showed the prognostic ability in HCC (P < 0.001 for OS/TTR). Furthermore, expression levels of Th17 associated cytokines including IL-6, -22, -17R and TNF-α were increased in serum of HCC patients compared to haemangioma patients. Importantly, activated human hepatic stellate cells induced in vitro expansion of IL-17+ CD4+ T cells.ConclusionsHigh expression of IL-17 and IL-17RE were promising predictors for poor outcome of HCC patients. The protumor power of IL-17 producing CD4+ T cells was probably involved in the crosstalk with different types of inflammatory/immune cells in HCC.