UniformAugment: A Search-free Probabilistic Data Augmentation Approach

LingChen, Tom Ching; Khonsari, Ava; Lashkari, Amirreza; Nazari, Mina Rafi; Sambee, Jaspreet Singh; Nascimento, Mario A.

doi:10.48550/arxiv.2003.14348

Cited by 15 publications

(48 citation statements)

References 17 publications

(38 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To study the limitation of negative sample free self-supervised methods and look into the boundary of momentum updating for the encoder, we further strengthen the magnitudes of risky augmentations for BYOL. In particu- lar, UA [17] and RA with more augmentation policies and higher distortion magnitudes are selected. The results presented in Tab.…”

Section: Main Results and Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Directional Self-supervised Learning for Heavy Image Augmentations

Bai¹,

Yang²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Only a few cherry-picked robust augmentation policies are beneficial to standard self-supervised image representation learning, despite the large augmentation family. In this paper, we propose a directional self-supervised learning paradigm (DSSL), which is compatible with significantly more augmentations. Specifically, we adapt risky augmentation policies after standard views augmented by robust augmentations, to generate harder risky view (RV). The risky view usually has a higher deviation from the original image than the standard robust view (SV). Unlike previous methods equally pairing all augmented views for symmetrical self-supervised training to maximize their similarities, DSSL treats augmented views of the same instance as a partially ordered set (SV↔SV, SV←RV), and then equips directional objective functions respecting to the derived relationships among views. DSSL can be easily implemented with a few lines of Pseudocode and is highly flexible to popular self-supervised learning frameworks, including Sim-CLR, SimSiam, BYOL. The extensive experimental results on CIFAR and ImageNet demonstrated that DSSL can stably improve these frameworks with compatibility to a wider range of augmentations.

show abstract

Section: Main Results and Discussionmentioning

confidence: 99%

“…The number of grids for JigSaw(n) is n × n. RA(m, n) is the RandAugment [6] with n augmentation transformation of m magnitude. UA denotes the UniformAumgent [17]. Right: validation accuracy of kNN classification during pre-training.…”

Section: Introductionmentioning

confidence: 99%

Directional Self-supervised Learning for Heavy Image Augmentations

Bai¹,

Yang²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Meanwhile, there are two approaches to textual data augmentation: data warping and synthetic over-sampling [60]. Recently, various methods have been proposed to search for augmentation policies for different tasks using reinforcement learning [11], and various other improved algorithms [37,38].…”

Section: Data Augmentationmentioning

confidence: 99%

Man versus Machine: AutoML and Human Experts' Role in Phishing Detection

Purwanto¹,

Pal²,

Blair³

et al. 2021

Preprint

View full text Add to dashboard Cite

Machine learning (ML) has developed rapidly in the past few years and has successfully been utilized for a broad range of tasks, including phishing detection. However, building an effective ML-based detection system is not a trivial task, and requires data scientists with knowledge of the relevant domain. Automated Machine Learning (AutoML) frameworks have received a lot of attention in recent years, enabling non-ML experts in building a machine learning model. This brings to an intriguing question of whether AutoML can outperform the results achieved by human data scientists. Our paper compares the performances of six well-known, state-of-the-art AutoML frameworks on ten different phishing datasets to see whether AutoML-based models can outperform manually crafted machine learning models. Our results indicate that AutoML-based models are able to outperform manually developed machine learning models in complex classification tasks, specifically in datasets where the features are not quite discriminative, and datasets with overlapping classes or relatively high degrees of non-linearity. Challenges also remain in building a real-world phishing detection system using AutoML frameworks due to the current support only on supervised classification problems, leading to the need for labeled data, and the inability to update the AutoML-based models incrementally. This indicates that experts with knowledge in the domain of phishing and cybersecurity are still essential in the loop of the phishing detection pipeline.

show abstract

“…The optimal transformation is dependent on the dataset. UniformAugment [8] augments images by uniformly sampling from the continuous space of augmentation transformations, hence, avoiding the costly search process for finding augmentation policies. Laskin et al [25] use two new augmentation schemes of random translation as well as random amplitude scaling alongside standard augmentation techniques (crop, cutout, flip, rotate, etc.).…”

Section: Data Augmentation For Enhancing Performancementioning

confidence: 99%

“…Data augmentation is an effective technique, often used for image classification to increase dataset size, help the model learn invariance, and regularize the model [8]. TUTOR targets non-image datasets.…”

Section: Introductionmentioning

confidence: 99%

TUTOR: Training Neural Networks Using Decision Rules as Model Priors

Hassantabar¹,

Terway²,

Jha³

2020

Preprint

View full text Add to dashboard Cite

The human brain has the ability to carry out new tasks with limited experience. It utilizes prior learning experiences to adapt the solution strategy to new domains. On the other hand, deep neural networks (DNNs) generally need large amounts of data and computational resources for training. However, this requirement is not met in many settings. To address these challenges, we propose the TUTOR DNN synthesis framework. TUTOR targets non-image datasets. It synthesizes accurate DNN models with limited available data, and reduced memory and computational requirements. It consists of three sequential steps: (1) drawing synthetic data from the same probability distribution as the training data and labeling the synthetic data based on a set of rules extracted from the real dataset, (2) use of two training schemes that combine synthetic data and training data to learn DNN weights, and (3) employing a grow-and-prune synthesis paradigm to learn both the weights and the architecture of the DNN to reduce model size while ensuring its accuracy. We show that in comparison with fully-connected DNNs, on an average TUTOR reduces the need for data by 6.0× (geometric mean), improves accuracy by 3.6%, and reduces the number of parameters (floating-point operations) by 4.7× (4.3×) (geometric mean). Thus, TUTOR is a less data-hungry, accurate, and efficient DNN synthesis framework.

show abstract

UniformAugment: A Search-free Probabilistic Data Augmentation Approach

Cited by 15 publications

References 17 publications

Directional Self-supervised Learning for Heavy Image Augmentations

Directional Self-supervised Learning for Heavy Image Augmentations

Man versus Machine: AutoML and Human Experts' Role in Phishing Detection

TUTOR: Training Neural Networks Using Decision Rules as Model Priors

Contact Info

Product

Resources

About