Large image datasets: A pyrrhic win for computer vision?

Prabhu, Vinay Uday; Birhane, Abeba

doi:10.48550/arxiv.2006.16923

Cited by 22 publications

(23 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1 The images were subjected to an array of automated filters designed to remove potentially offensive content. While certainly not perfect, this substantially reduces the issues that plague other large image datasets [8,55]. We construct a multi-label dataset using these images by converting all hashtags into their corresponding canonical targets (note that a single image may have multiple hashtags).…”

Section: Hashtag Dataset Collectionmentioning

confidence: 99%

Revisiting Weakly Supervised Pre-Training of Visual Perception Models

Singh¹,

Gustafson²,

Adcock³

et al. 2022

Preprint

View full text Add to dashboard Cite

Model pre-training is a cornerstone of modern visual recognition systems. Although fully supervised pre-training on datasets like ImageNet is still the de-facto standard, recent studies suggest that large-scale weakly supervised pretraining can outperform fully supervised approaches. This paper revisits weakly-supervised pre-training of models using hashtag supervision with modern versions of residual networks and the largest-ever dataset of images and corresponding hashtags. We study the performance of the resulting models in various transfer-learning settings including zero-shot transfer. We also compare our models with those obtained via large-scale self-supervised learning. We find our weakly-supervised models to be very competitive across all settings, and find they substantially outperform their self-supervised counterparts. We also include an investigation into whether our models learned potentially troubling associations or stereotypes. Overall, our results provide a compelling argument for the use of weakly supervised learning in the development of visual recognition systems. Our models, Supervised Weakly through hashtAGs (SWAG), are available publicly.

show abstract

Section: Hashtag Dataset Collectionmentioning

confidence: 99%

Revisiting Weakly Supervised Pre-Training of Visual Perception Models

Singh¹,

Gustafson²,

Adcock³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In the supplementary material we include experiments with an out-of-training face dataset Kärkkäinen & Joo (2019). Although we are aware of the ethical issues with ImageNet, and share the concerns over its nonconsensual content Prabhu & Birhane (2020), a direct comparison to existing results in the literature requires us to use the dataset.…”

Section: Data Preprocessingmentioning

confidence: 99%

Comparing Deep Neural Nets with UMAP Tour

Li,

Scheidegger

2021

Preprint

View full text Add to dashboard Cite

Neural networks should be interpretable to humans. In particular, there is a growing interest in concepts learned in a layer and similarity between layers. In this work, a tool, UMAP Tour, is built to visually inspect and compare internal behavior of realworld neural network models using well-aligned, instance-level representations. The method used in the visualization also implies a new similarity measure between neural network layers. Using the visual tool and the similarity measure, we find concepts learned in state-of-the-art models and dissimilarities between them, such as GoogLeNet and ResNet.Preprint. Under review.

show abstract

“…Note, to our knowledge, these datasets are not known to contain personally identifiable information or offensive content. Although CIFAR-10 and CINIC-100 use images from the problematic ImageNet and Tiny Images [32], they contain manually selected subsets. The list of dataset-model combinations, or tasks, available in the trained model corpus can be seen in the first two rows of Table 1.…”

Section: Generalization Predictions: Experimental Setupmentioning

confidence: 99%

Predicting Deep Neural Network Generalization with Perturbation Response Curves

Schiff¹,

Quanz²,

Das³

et al. 2021

Preprint

View full text Add to dashboard Cite

The field of Deep Learning is rich with empirical evidence of human-like performance on a variety of prediction tasks. However, despite these successes, the recent Predicting Generalization in Deep Learning (PGDL) NeurIPS 2020 competition [1] suggests that there is a need for more robust and efficient measures of network generalization. In this work, we propose a new framework for evaluating the generalization capabilities of trained networks. We use perturbation response (PR) curves that capture the accuracy change of a given network as a function of varying levels of training sample perturbation. From these PR curves, we derive novel statistics that capture generalization capability. Specifically, we introduce two new measures for accurately predicting generalization gaps: the Gi-score and Pal-score, that are inspired by the Gini coefficient and Palma ratio (measures of income inequality), that accurately predict generalization gaps. Using our framework applied to intra and inter class sample mixup, we attain better predictive scores than the current state-of-the-art measures on a majority of tasks in the PGDL competition. In addition, we show that our framework and the proposed statistics can be used to capture to what extent a trained network is invariant to a given parametric input transformation, such as rotation or translation. Therefore, these generalization gap prediction statistics also provide a useful means for selecting the optimal network architectures and hyperparameters that are invariant to a certain perturbation.Preprint. Under review.

show abstract

Large image datasets: A pyrrhic win for computer vision?

Cited by 22 publications

References 36 publications

Revisiting Weakly Supervised Pre-Training of Visual Perception Models

Revisiting Weakly Supervised Pre-Training of Visual Perception Models

Comparing Deep Neural Nets with UMAP Tour

Predicting Deep Neural Network Generalization with Perturbation Response Curves

Contact Info

Product

Resources

About