SpinalNet: Deep Neural Network with Gradual Input

Kabir, H M Dipu; Abdar, Moloud; Jalali, Seyed Mohammad Jafar; Khosravi, Abbas; Atiya, Amir F.; Nahavandi, Saeid; Srinivasan, Dipti

doi:10.48550/arxiv.2007.03347

Cited by 12 publications

(22 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Predictions made without UQ are usually not trustworthy and inaccurate. To understand the Deep Learning (DL) [11], [12] process life cycle, we need to comprehend the role of UQ in DL. The DL models start with the collection of most comprehensive and potentially relevant datasets available for decision making process.…”

Section: Epistemic Aleatoricmentioning

confidence: 99%

“…p(y * |x * , X, Y ) ≈ p(y * |x * , ω)q * θ (ω)dω =: q * θ (y * , x * ), (10) where q * θ (ω) indicates the optimized objective. KL divergence minimization can also be rearranged into the evidence lower bound (ELBO) maximization [21]: L V I (θ) := q θ (ω) log p(Y |X, ω)dω − KL(q θ (ω) p(ω)), (11) where q θ (ω) is able to describe the data well by maximizing the first term, and be as close as possible to the prior by minimizing the second term. This process is called variational inference (VI).…”

Section: Uncertainty Modelingmentioning

confidence: 99%

See 1 more Smart Citation

A review of uncertainty quantification in deep learning: Techniques, applications and challenges

et al. 2021

Self Cite

View full text Add to dashboard Cite

Section: Epistemic Aleatoricmentioning

confidence: 99%

Section: Uncertainty Modelingmentioning

confidence: 99%

A review of uncertainty quantification in deep learning: Techniques, applications and challenges

et al. 2021

Self Cite

View full text Add to dashboard Cite

“…We choose pre-trained model ResNet50 for feature extraction in the STL10 experiment. The classifier used for STL10 is SpinalNet [Kabir et al, 2020] which is a powerful network imitating human brachial plexus. We choose SGD as the optimizer.…”

Section: Experiments Setupmentioning

confidence: 99%

Self-Evolutionary Clustering

Wang¹,

Lü²,

Liu³

2022

Preprint

View full text Add to dashboard Cite

Deep clustering outperforms conventional clustering by mutually promoting representation learning and cluster assignment. However, most existing deep clustering methods suffer from two major drawbacks. First, most cluster assignment methods are based on simple distance comparison and highly dependent on the target distribution generated by a handcrafted nonlinear mapping. These facts largely limit the possible performance that deep clustering methods can reach. Second, the clustering results can be easily guided towards wrong direction by the misassigned samples in each cluster. The existing deep clustering methods are incapable of discriminating such samples. To address these issues, a novel modular Self-Evolutionary Clustering (Self-EvoC) framework is constructed, which boosts the clustering performance by classification in a self-supervised manner. Fuzzy theory is used to score the sample membership with probability which evaluates the intermediate clustering result certainty of each sample. Based on which, the most reliable samples can be selected and augmented. The augmented data are employed to fine-tune an off-theshelf deep network classifier with the labels from the clustering, which results in a model to generate the target distribution. The proposed framework can efficiently discriminate sample outliers and generate better target distribution with the assistance of self-supervised classifier. Extensive experiments indicate that the Self-EvoC remarkably outperforms state-of-the-art deep clustering methods on three benchmark datasets.

show abstract

“…Here we implement each data set with pre-trained models with a small configuration. This change is to replace the last fully connected layer in each model with the SpinalNet [39] layer. SpinalNet is a novel network that is inspired from the human somatosensory system [40].…”

Section: Phase 4 Investigating Spinalnetmentioning

confidence: 99%

A Comprehensive Study on Torchvision Pre-trained Models for Fine-grained Inter-species Classification

Albardi¹,

Kabir²,

Islam³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

This study aims to explore different pre-trained models offered in the Torchvision package which is available in the PyTorch library. And investigate their effectiveness on finegrained images classification. Transfer Learning is an effective method of achieving extremely good performance with insufficient training data. In many real-world situations, people cannot collect sufficient data required to train a deep neural network model efficiently. Transfer Learning models are pre-trained on a large data set, and can bring a good performance on smaller datasets with significantly lower training time. Torchvision package offers us many models to apply the Transfer Learning on smaller datasets. Therefore, researchers may need a guideline for the selection of a good model. We investigate Torchvision pretrained models on four different data sets: 10 Monkey Species, 225 Bird Species, Fruits 360, and Oxford 102 Flowers. These data sets have images of different resolutions, class numbers, and different achievable accuracies. We also apply their usual fullyconnected layer and the Spinal fully-connected layer to investigate the effectiveness of SpinalNet. The Spinal fully-connected layer brings better performance in most situations. We apply the same augmentation for different models for the same data set for a fair comparison. This paper may help future Computer Vision researchers in choosing a proper Transfer Learning model.

show abstract

SpinalNet: Deep Neural Network with Gradual Input

Cited by 12 publications

References 0 publications

A review of uncertainty quantification in deep learning: Techniques, applications and challenges

A review of uncertainty quantification in deep learning: Techniques, applications and challenges

Self-Evolutionary Clustering

A Comprehensive Study on Torchvision Pre-trained Models for Fine-grained Inter-species Classification

Contact Info

Product

Resources

About