3D Common Corruptions and Data Augmentation

Kar, Oğuzhan Fatih; Yeo, Teresa; Atanov, Andrei; Zamir, Amir

doi:10.1109/cvpr52688.2022.01839

Cited by 30 publications

(9 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, the value v i of a i less than 26 is set to 0. Taking into account the above two reasons we construct the value criteria as Equation (1), which can adaptively adjust with the size and number of the objects.…”

Section: Establishment Of Object Value Criteriamentioning

confidence: 99%

See 1 more Smart Citation

Value-Guided Adaptive Data Augmentation for Imbalanced Small Object Detection

Wang,

Sui,

Jiang

et al. 2024

Electronics

View full text Add to dashboard Cite

Data augmentation is considered a promising technique to resolve the imbalance of large and small objects. Unfortunately, most existing methods augment all small objects indiscriminately, regardless of their learnability and proportion. This tends to result in wasteful enlargement for many weak, low-information objects but under-augmentation for rare and learnable objects. To this end, we propose a value-guided adaptive data augmentation for scale- and proportion-imbalanced small object detection (ValCopy-Paste). Specifically, we first develop a non-learning object value criteria to determine whether one object should be expanded. Both scale-based learnability and quantity-based necessity are involved in this criteria. Then, the value distribution of objects in the dataset can be further constructed on the basis of the relevant object values. This helps to ensure that those uncommon, learnable objects that deserve enhancement are more likely to be enhanced. Additionally, we propose to enhance the data by pasting the sampled objects into relatively smooth portions of fresh background images, rather than arbitrary areas of any background images. This helps to boost data diversity while reducing the interference from complicated backgrounds. Evidently, our method does not require sophisticated training and just depends on the size and distribution of the objects in the dataset. Extensive experiments on MS COCO 2017 and PASCAL VOC 2012 demonstrate that our method achieves better performance than state-of-the-art methods.

show abstract

Section: Establishment Of Object Value Criteriamentioning

confidence: 99%

“…As an essential step for traffic surveillance and maritime rescue, object detection has experienced tremendous progress [1][2][3][4][5][6][7][8]. This is not only due to the powerful representation ability of deep neural networks but it is also reliant on massive training data [9][10][11][12].…”

Section: Introductionmentioning

confidence: 99%

Value-Guided Adaptive Data Augmentation for Imbalanced Small Object Detection

Wang,

Sui,

Jiang

et al. 2024

Electronics

View full text Add to dashboard Cite

show abstract

“…Xiao et al [9], Sagawa et al [18] investigate whether models are biased towards background cues by compositing foreground objects with various background images (IMAGENET-9, WATERBIRDS). Extending upon [16], Kar et al [19] introduce corruptions that capture 3D information. They aim to guard against natural corruptions, such as camera rotation, camera focus change, motion blur.…”

Section: Related Workmentioning

confidence: 99%

Benchmarking Robustness to Adversarial Image Obfuscations

Stimberg¹,

Chakrabarti²,

Lu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Automated content filtering and moderation is an important tool that allows online platforms to build striving user communities that facilitate cooperation and prevent abuse. Unfortunately, resourceful actors try to bypass automated filters in a bid to post content that violate platform policies and codes of conduct. To reach this goal, these malicious actors may obfuscate policy violating images (e.g. overlay harmful images by carefully selected benign images or visual patterns) to prevent machine learning models from reaching the correct decision. In this paper, we invite researchers to tackle this specific issue and present a new image benchmark. This benchmark, based on IMAGENET, simulates the type of obfuscations created by malicious actors. It goes beyond IMAGENET-C and IMAGENET-C by proposing general, drastic, adversarial modifications that preserve the original content intent. It aims to tackle a more common adversarial threat than the one considered by pnorm bounded adversaries. We evaluate 33 pretrained models on the benchmark and train models with different augmentations, architectures and training methods on subsets of the obfuscations to measure generalization. We hope this benchmark will encourage researchers to test their models and methods and try to find new approaches that are more robust to these obfuscations.

show abstract

“…Under such a setting, we are able to estimate the overall robustness of models against corruption. In the following, benchmark studies and techniques across various computer vision tasks are booming (Kar et al, 2022;Michaelis et al, 2019;Kamann & Rother, 2020). These studies on corruption robustness bridge the gap between research in well-setup lab environments and deployment in the field.…”

Section: Problem Settingmentioning

confidence: 99%

“…For example, weather changes like rain and fog, and data pre-processing like saturate adjustment and compression can corrupt the test data. Many works show that the common corruptions arising in nature can degrade the performance of models at test time significantly (Hendrycks & Dietterich, 2019;Yi et al, 2021;Kar et al, 2022;Geirhos et al, 2018;. In video classification, Yi et al (2021) demonstrates the vulnerability of models against corruptions like noise, blur, and weather variations.…”

Section: Introductionmentioning

confidence: 99%

Temporal Coherent Test-Time Optimization for Robust Video Classification

Yi¹,

Yang²,

Wang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Deep neural networks are likely to fail when the test data is corrupted in realworld deployment (e.g., blur, weather, etc.). Test-time optimization is an effective way that adapts models to generalize to corrupted data during testing, which has been shown in the image domain. However, the techniques for improving video classification corruption robustness remain few. In this work, we propose a Temporal Coherent Test-time Optimization framework (TeCo) to utilize spatiotemporal information in test-time optimization for robust video classification. To exploit information in video with self-supervised learning, TeCo minimizes the entropy of the prediction based on the global content from video clips. Meanwhile, it also feeds local content to regularize the temporal coherence at the feature level. TeCo retains the generalization ability of various video classification models and achieves significant improvements in corruption robustness across Mini Kinetics-C and Mini SSV2-C. Furthermore, TeCo sets a new baseline in video classification corruption robustness via test-time optimization.

show abstract

3D Common Corruptions and Data Augmentation

Cited by 30 publications

References 37 publications

Value-Guided Adaptive Data Augmentation for Imbalanced Small Object Detection

Value-Guided Adaptive Data Augmentation for Imbalanced Small Object Detection

Benchmarking Robustness to Adversarial Image Obfuscations

Temporal Coherent Test-Time Optimization for Robust Video Classification

Contact Info

Product

Resources

About