Improving Whole-Brain Neural Decoding of fMRI with Domain Adaptation

Zhou, Shuo; Cox, Christopher R.; Lu, Haiping

doi:10.1007/978-3-030-32692-0_31

Cited by 6 publications

(4 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among the surveyed 31 symmetric approaches, direct approaches operated on the feature representations across domains by minimizing their differences (via mutual information [ 63 ], maximum mean discrepancy [ 46 , 49 , 64 ], Euclidean distance [ 65 , 66 , 67 , 68 , 69 , 70 , 71 ], Wasserstein distance [ 72 ], and average likelihood [ 73 ]), maximizing their correlation [ 74 , 75 ] or covariance [ 36 ], and introducing sparsity with L1/L2 norms [ 42 , 76 ]. On the other hand, indirect approaches were applied via adversarial training [ 28 , 41 , 54 , 77 , 78 , 79 , 80 , 81 , 82 , 83 , 84 , 85 ], and knowledge distillation [ 86 ].…”

Section: Resultsmentioning

confidence: 99%

“…The most common approach to apply a prior-sharing strategy—and, in general, transfer learning—was fine-tuning all the parameters of a pretrained CNN [ 29 , 31 , 32 , 33 , 35 , 39 , 71 , 87 , 88 , 89 , 90 , 91 , 92 , 93 , 94 , 95 , 96 , 97 , 98 , 99 , 100 , 101 , 102 , 103 , 104 , 105 , 106 , 107 , 108 , 109 , 110 , 111 , 112 , 113 , 114 , 115 , 116 , 117 , 118 , 119 ] (80% of all prior-sharing methods). Other approaches utilized Bayesian graphical models [ 37 , 38 , 120 , 121 ], graph neural networks [ 122 ], kernel methods [ 64 , 123 ], multilayer perceptrons [ 124 ], and Pearson-correlation methods [ 125 ]. Additionally, Sato et al […”

Section: Resultsmentioning

confidence: 99%

“…The most common approach to apply a prior-sharing strategy-and, in general, transfer learning-was fine-tuning all the parameters of a pretrained CNN [29,[31][32][33]35,39,71, (80% of all prior-sharing methods). Other approaches utilized Bayesian graphical models [37,38,120,121], graph neural networks [122], kernel methods [64,123], multilayer perceptrons [124], and Pearson-correlation methods [125]. Additionally, Sato et al [27] proposed a general framework to inhibit negative transfer.…”

Section: Parameter-based Approachesmentioning

confidence: 99%

See 2 more Smart Citations

Transfer Learning in Magnetic Resonance Brain Imaging: A Systematic Review

et al. 2021

View full text Add to dashboard Cite

(1) Background: Transfer learning refers to machine learning techniques that focus on acquiring knowledge from related tasks to improve generalization in the tasks of interest. In magnetic resonance imaging (MRI), transfer learning is important for developing strategies that address the variation in MR images from different imaging protocols or scanners. Additionally, transfer learning is beneficial for reutilizing machine learning models that were trained to solve different (but related) tasks to the task of interest. The aim of this review is to identify research directions, gaps in knowledge, applications, and widely used strategies among the transfer learning approaches applied in MR brain imaging; (2) Methods: We performed a systematic literature search for articles that applied transfer learning to MR brain imaging tasks. We screened 433 studies for their relevance, and we categorized and extracted relevant information, including task type, application, availability of labels, and machine learning methods. Furthermore, we closely examined brain MRI-specific transfer learning approaches and other methods that tackled issues relevant to medical imaging, including privacy, unseen target domains, and unlabeled data; (3) Results: We found 129 articles that applied transfer learning to MR brain imaging tasks. The most frequent applications were dementia-related classification tasks and brain tumor segmentation. The majority of articles utilized transfer learning techniques based on convolutional neural networks (CNNs). Only a few approaches utilized clearly brain MRI-specific methodology, and considered privacy issues, unseen target domains, or unlabeled data. We proposed a new categorization to group specific, widely-used approaches such as pretraining and fine-tuning CNNs; (4) Discussion: There is increasing interest in transfer learning for brain MRI. Well-known public datasets have clearly contributed to the popularity of Alzheimer’s diagnostics/prognostics and tumor segmentation as applications. Likewise, the availability of pretrained CNNs has promoted their utilization. Finally, the majority of the surveyed studies did not examine in detail the interpretation of their strategies after applying transfer learning, and did not compare their approach with other transfer learning approaches.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Parameter-based Approachesmentioning

confidence: 99%

See 1 more Smart Citation

Transfer Learning in Magnetic Resonance Brain Imaging: A Systematic Review

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Insufficient training data and domain-adaptation [17] are the focus of many recent machine learning works, including transfer learning [18], unsupervised and self-supervised learning [19,20,21,22], semi-supervised and transductive learning [23,24,25,26]. Nevertheless, these approaches often assume that there is sufficient labeled data in the 'source domain', and that the problem is the generalization to the unlabeled 'target domain'.…”

Section: Introductionmentioning

confidence: 99%

From voxels to pixels and back: Self-supervision in natural-image reconstruction from fMRI

Beliy,

Gaziv,

Hoogi

et al. 2019

Preprint

View full text Add to dashboard Cite

Reconstructing observed images from fMRI brain recordings is challenging. Unfortunately, acquiring sufficient "labeled" pairs of {Image, fMRI} (i.e., images with their corresponding fMRI responses) to span the huge space of natural images is prohibitive for many reasons. We present a novel approach which, in addition to the scarce labeled data (training pairs), allows to train fMRI-to-image reconstruction networks also on "unlabeled" data (i.e., images without fMRI recording, and fMRI recording without images). The proposed model utilizes both an Encoder network (image-to-fMRI) and a Decoder network (fMRI-to-image). Concatenating these two networks back-to-back (Encoder-Decoder & Decoder-Encoder) allows augmenting the training with both types of unlabeled data. Importantly, it allows training on the unlabeled test-fMRI data. This self-supervision adapts the reconstruction network to the new input test-data, despite its deviation from the statistics of the scarce training data. 2 Project Website: http://www.wisdom.weizmann.ac.il/~vision/ssfmri2im/ * Equal contribution 2 We will make our code publicly available upon publication.Preprint. Under review.

show abstract