X-CNN: Cross-modal convolutional neural networks for sparse datasets

Veličković, Petar; Wang, Duo; Laney, Nicholas D.; Lió, Píetro

doi:10.1109/ssci.2016.7849978

Cited by 22 publications

(29 citation statements)

References 18 publications

(18 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this paper, we developed a feature extractor sub-network (referred to as the multi-modal feature extractor in fig. 2), inspired by the parameter-efficient separable and grouped convolutional layers presented in AlexNet (Krizhevsky et al, 2012) and Xception (Chollet, 2017, Velickovic et al, 2016. In detail, the layers of the feature extractor are shared between two tasks -MCIto-AD conversion prediction and AD/HC classification (see fig.…”

Section: Architecture Overviewmentioning

confidence: 99%

A parameter-efficient deep learning approach to predict conversion from mild cognitive impairment to Alzheimer’s disease

Spasov¹,

Passamonti²,

Duggento

et al. 2018

Preprint

View full text Add to dashboard Cite

Some forms of mild cognitive impairment (MCI) can be the clinical precursor of severe dementia like Alzheimer's disease (AD), while other types of MCI tend to remain stable over-time and do not progress to AD pathology. To choose an effective and personalized treatment for AD, we need to identify which MCI patients are at risk of developing AD and which are not. Here, we present a novel deep learning architecture, based on dual learning and an ad hoc layer for 3D separable convolutions, which aims at identifying those people with MCI who have a high likelihood of developing AD. Our deep learning procedures combine structural magnetic resonance imaging (MRI), demographic, neuropsychological, and APOe4 genotyping data as input measures. The most novel characteristics of our machine learning model compared to previous ones are as follows: 1) multi-tasking, in the sense that our deep learning model jointly learns to simultaneously predict both MCI to AD conversion, and AD vs healthy classification which facilitates the relevant feature extraction for prognostication; 2) the neural network classifier employs relatively few parameters compared to other deep learning architectures (we use ~550,000 network parameters, orders of magnitude lower than other network designs) without compromising network complexity and hence significantly limits data-overfitting; 3) both structural MRI images and warp field characteristics, which quantify the amount of volumetric change compared to the common template, were used as separate input streams to extract as much information as possible from the MRI data. All the analyses were performed on a subset of the Alzheimer's Disease Neuroimaging Initiative (ADNI) database, for a total of n=785 participants (192 AD, 409 MCI, and184 healthy controls (HC)). We found that the most predictive combination of inputs included the structural MRI images and the demographic, neuropsychological, and APOe4 data, while the warp field metric added little predictive value. We achieved an area under the ROC curve (AUC) of 0.925 with a 10-fold cross-validated accuracy of 86%, a sensitivity of 87.5% and specificity of 85% in classifying MCI patients who developed AD in three years' time from those individuals showing stable MCI over the same time-period. To the best of our knowledge, this is the highest performance reported on a test set achieved in the literature using similar data. The same network provided an AUC of 1 and 100% accuracy, sensitivity and specificity when classifying NC from AD. We also demonstrated that our classification framework was robust to different co-registration templates and possibly irrelevant features / image sections. Our approach is flexible and can in principle integrate other imaging modalities, such as PET, and a more diverse group of clinical data. The convolutional framework is potentially applicable to any 3D image dataset and gives the flexibility to design a computer-aided diagnosis system targeting the prediction of any medical condition utilizing multi-modal imaging a...

show abstract

Section: Architecture Overviewmentioning

confidence: 99%

A parameter-efficient deep learning approach to predict conversion from mild cognitive impairment to Alzheimer’s disease

Spasov¹,

Passamonti²,

Duggento

et al. 2018

Preprint

View full text Add to dashboard Cite

show abstract

“…From a computational perspective, novel solutions must be devised to combine multi-modal imaging data [79]. In the case of DNNs, a topology explicitly designed for information exchange-between sub-networks processing the data from a single modality-through cross-connections, such as in the case of cross-modal CNNs (X-CNNs) [80], might be suitable for combining multi-modal imaging data.…”

Section: Discussionmentioning

confidence: 99%

A Hybrid End-to-End Approach Integrating Conditional Random Fields into CNNs for Prostate Cancer Detection on MRI

et al. 2020

View full text Add to dashboard Cite

Prostate Cancer (PCa) is the most common oncological disease in Western men. Even though a growing effort has been carried out by the scientific community in recent years, accurate and reliable automated PCa detection methods on multiparametric Magnetic Resonance Imaging (mpMRI) are still a compelling issue. In this work, a Deep Neural Network architecture is developed for the task of classifying clinically significant PCa on non-contrast-enhanced MR images. In particular, we propose the use of Conditional Random Fields as a Recurrent Neural Network (CRF-RNN) to enhance the classification performance of XmasNet, a Convolutional Neural Network (CNN) architecture specifically tailored to the PROSTATEx17 Challenge. The devised approach builds a hybrid end-to-end trainable network, CRF-XmasNet, composed of an initial CNN component performing feature extraction and a CRF-based probabilistic graphical model component for structured prediction, without the need for two separate training procedures. Experimental results show the suitability of this method in terms of classification accuracy and training time, even though the high-variability of the observed results must be reduced before transferring the resulting architecture to a clinical environment. Interestingly, the use of CRFs as a separate postprocessing method achieves significantly lower performance with respect to the proposed hybrid end-to-end approach. The proposed hybrid end-to-end CRF-RNN approach yields excellent peak performance for all the CNN architectures taken into account, but it shows a high-variability, thus requiring future investigation on the integration of CRFs into a CNN.

show abstract

“…Experiments have shown that a fully-connected graph allows learning appropriate connections between each pair of modalities and sharing information between all. This leads to better performance than a more restricted variant used in XKerasNet and XFitNet [4].…”

Section: Connectivity (Step 7)mentioning

confidence: 99%

Automatic Inference of Cross-Modal Connection Topologies for X-CNNs

Karazija

Veličković

Lió

2018

Advances in Neural Networks – ISNN 2018

View full text Add to dashboard Cite

This paper introduces a way to learn cross-modal convolutional neural network (X-CNN) architectures from a base convolutional network (CNN) and the training data to reduce the design cost and enable applying cross-modal networks in sparse data environments. Two approaches for building X-CNNs are presented. The base approach learns the topology in a data-driven manner, by using measurements performed on the base CNN and supplied data. The iterative approach performs further optimisation of the topology through a combined learning procedure, simultaneously learning the topology and training the network. The approaches were evaluated agains examples of hand-designed X-CNNs and their base variants, showing superior performance and, in some cases, gaining an additional 9% of accuracy. From further considerations, we conclude that the presented methodology takes less time than any manual approach would, whilst also significantly reducing the design complexity. The application of the methods is fully automated and implemented in Xsertion library. 1

show abstract

X-CNN: Cross-modal convolutional neural networks for sparse datasets

Cited by 22 publications

References 18 publications

A parameter-efficient deep learning approach to predict conversion from mild cognitive impairment to Alzheimer’s disease

A parameter-efficient deep learning approach to predict conversion from mild cognitive impairment to Alzheimer’s disease

A Hybrid End-to-End Approach Integrating Conditional Random Fields into CNNs for Prostate Cancer Detection on MRI

Automatic Inference of Cross-Modal Connection Topologies for X-CNNs

Contact Info

Product

Resources

About