Deep learning models can be applied successfully in real-work problems; however, training most of these models requires massive data. Recent methods use language and vision, but unfortunately, they rely on datasets that are not usually publicly available. Here we pave the way for further research in the multimodal language-vision domain for radiology. In this paper, we train a representation learning method that uses local and global representations of the language and vision through an attention mechanism and based on the publicly available Indiana University Radiology Report (IU-RR) dataset. Furthermore, we use the learned representations to diagnose five lung pathologies: atelectasis, cardiomegaly, edema, pleural effusion, and consolidation. Finally, we use both supervised and zero-shot classifications to extensively analyze the performance of the representation learning on the IU-RR dataset. Average Area Under the Curve (AUC) is used to evaluate the accuracy of the classifiers for classifying the five lung pathologies. The average AUC for classifying the five lung pathologies on the IU-RR test set ranged from 0.85 to 0.87 using the different training datasets, namely CheXpert and CheXphoto. These results compare favorably to other studies using UI-RR. Extensive experiments confirm consistent results for classifying lung pathologies using the multimodal global local representations of language and vision information.
Alterations in brain network connectivity play an important role in the pathogenesis of schizophrenia. We investigate whether large-scale Kernelized Granger Causality (lsKGC) can capture such alterations using restingstate fMRI data. Our method utilizes dimension reduction combined with the augmentation of source timeseries in a predictive time-series model for estimating directed causal relationships among fMRI time-series. As a multivariate approach, lsKGC identifies the relationship of the underlying dynamic system in the presence of all other time-series. Here, we examine the ability of lsKGC to accurately identify schizophrenia patients from fMRI data using a subset of 31 subjects from the Centers of Biomedical Research Excellence (COBRE) data repository. We use brain connections estimated by lsKGC as features for classification. After feature extraction, we perform feature selection by Kendall's tau rank correlation coefficient followed by classification using a support vector machine. For reference, we compare our results with cross-correlation, typically used in the literature as a standard measure of functional connectivity, and several other standard methods. Using 100 different training/test data splits with 10-fold cross-validation we obtain mean/std f1-scores of 84.87% ± 19.78% and mean Area Under the receiver operating characteristic Curve (AUC) values of 93.00% ± 16.61% across all tested numbers of features for lsKGC, which is significantly better than the results obtained with cross-correlation (AUC=53.25% ± 29.29%, f1-score=45.03% ± 30.82%), and multiple other competing methods,
The literature suggests that schizophrenia is associated with alterations in brain network connectivity. We investigate whether large-scale Augmented Granger Causality (lsAGC) can capture such alterations using restingstate fMRI data. Our method utilizes dimension reduction combined with the augmentation of source time-series in a predictive time-series model for estimating directed causal relationships among fMRI time-series. As a multivariate approach, lsAGC identifies the relationship of the underlying dynamic system in the presence of all other time-series. Here, we examine the ability of lsAGC to accurately identify schizophrenia patients from fMRI data using a subset of 31 subjects from the Centers of Biomedical Research Excellence (COBRE) data repository. We use brain connections estimated by lsAGC as features for classification. After feature extraction, we perform feature selection by Kendall's tau rank correlation coefficient followed by classification using a support vector machine. For reference, we compare our results with cross-correlation, typically used in the literature as a standard measure of functional connectivity, and several other standard methods. Using 30 different training/test data splits with 10-fold cross-validation we obtain mean/std f1-scores of 82.89% ± 17.25% and mean Area Under the receiver operating characteristic Curve (AUC) values of 93.33% ± 12.81% across all tested numbers of features for lsAGC, which is significantly better than the results obtained with cross-correlation (AUC=78.33%
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.