Learning in the machine: To share or not to share?

Ott, Jordan; Linstead, Erik; LaHaye, Nicholas; Baldi, Pierre

doi:10.1016/j.neunet.2020.03.016

Cited by 14 publications

(9 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(2020) , which may be applied in future studies, and generally makes it more comparable to that work. Second, because of the weight-sharing effects of convolutions, which reduce the total number of trainable parameters, it acts as a regularization method that builds on the dropout and batch normalization layers, which is appropriate for small training samples that may be prone to overfitting ( Kukačka, Golkov, Cremers, 2017 , Ott, Linstead, LaHaye, Baldi, 2020 ). To further validate the model, we additionally compare it to the performance of an ensemble of fully-connected neural networks lacking the convolutional layers.…”

Section: Methodsmentioning

confidence: 99%

Deep learning for sex classification in resting-state and task functional brain networks from the UK Biobank

Leming

Suckling

2021

NeuroImage

View full text Add to dashboard Cite

Highlights Applied deep learning to sex classification in UK BioBank fMRI connectomes. Deep learning classifies sex better in resting-state than in task fMRI. Algorithm to balance out multiple confounds from an fMRI dataset. Adapted two deep learning visualization methods to fMRI connectome classification. Analyzed role of three brain a priori networks in sex classification.

show abstract

Section: Methodsmentioning

confidence: 99%

Deep learning for sex classification in resting-state and task functional brain networks from the UK Biobank

Leming

Suckling

2021

NeuroImage

View full text Add to dashboard Cite

show abstract

“…This is easily achieved through data augmentation by translating each training image in all possible directions, something that may happen automatically in the real world due to moving objects, or head/eye motions. With this data augmentation, the weights of the convolution neurons remain similar throughout training, since they are trained on the same data, without any exact weight sharing [Ott et al, 2020]. This approach ought to be tried in Tourbillon.…”

Section: Discussionmentioning

confidence: 99%

Tourbillon: a Physically Plausible Neural Architecture

Tavakoli,

Sadowski,

Baldi

2021

Preprint

Self Cite

View full text Add to dashboard Cite

In a physical neural system, backpropagation is faced with a number of obstacles including: the need for labeled data, the violation of the locality learning principle, the need for symmetric connections, and the lack of modularity. Tourbillon is a new architecture that addresses all these limitations. At its core, it consists of a stack of circular autoencoders followed by an output layer. The circular autoencoders are trained in self-supervised mode by recirculation algorithms and the top layer in supervised mode by stochastic gradient descent, with the option of propagating error information through the entire stack using non-symmetric connections. While the Tourbillon architecture is meant primarily to address physical constraints, and not to improve current engineering applications of deep learning, we demonstrate its viability on standard benchmark datasets including MNIST, Fashion MNIST, and CIFAR10. We show that Tourbillon can achieve comparable performance to models trained with backpropagation and outperform models that are trained with other physically plausible algorithms, such as feedback alignment.Preprint. Under review.

show abstract

“…We can think of receptive field based neurons organized in a hierarchical architecture that carry out translation equivariance without sharing their weights. This is strongly motivated also by the arguable biological plausibility of the mechanism of weight sharing [76]. Such a lack of plausibility is more serious than the supposed lack of a truly local computational scheme in Backpropagation, which mostly comes from the lack of delay in the forward model of the neurons [14].…”

Section: Why Receptive Fields and Hierarchical Architectures?mentioning

confidence: 99%