Shift Equivariance in Object Detection

Manfredi, Marco; Wang, Yu

doi:10.48550/arxiv.2008.05787

Cited by 3 publications

(3 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One most relevant work [7] to ours is from the NLP field: the authors found universally transferable sparse matching subnetworks (at 40% to 90% sparsity), from the pre-trained initialization of BERT models. Finding their work inspiring, we stress that transplanting their NLP findings to our CV models is highly nontrivial due to multiple barriers: (1) pretraining BERT uses only a self-supervised objective called "masked language model" (MLM) [16], while pre-training CV models has a significant variety of popular options, ranging from the supervised fashion [40], to self-supervision yet with numerous objectives [17,34,8]; (2) BERT models consist of self-attention and fully-connected sub-layers, differing much from the standard convolutional architectures in CV; (3) further complicating the issue is that different CV downstream tasks are known to rely on different priors and invariances; for example, while classification often calls on shift invariance, detection assumes location shift equivariance [70,49]. That questions the feasibility of asking for one mask to transfer among them all.…”

Section: Related Workmentioning

confidence: 99%

The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models

Chen¹,

Frankle²,

Chang³

et al. 2020

Preprint

View full text Add to dashboard Cite

The computer vision world has been re-gaining enthusiasm in various pre-trained models, including both classical ImageNet supervised pre-training and recently emerged selfsupervised pre-training such as simCLR [8] and MoCo [34]. Pre-trained weights often boost a wide range of downstream tasks including classification, detection, and segmentation. Latest studies suggest that the pre-training benefits from gigantic model capacity [9]. We are hereby curious and ask: after pre-training, does a pre-trained model indeed have to stay large for its universal downstream transferability?In this paper, we examine the supervised and selfsupervised pre-trained models through the lens of lottery ticket hypothesis (LTH) [26]. LTH identifies highly sparse matching subnetworks that can be trained in isolation from (nearly) scratch, to reach the full models' performance. We extend the scope of LTH to questioning whether matching subnetworks still exist in the pre-training models, that enjoy the same downstream transfer performance. Our extensive experiments convey an overall positive message: from all pre-trained weights obtained by ImageNet classification, simCLR and MoCo, we are consistently able to locate such matching subnetworks at 59.04% to 96.48% sparsity that transfer universally to multiple downstream tasks, whose performance see no degradation compared to using full pretrained weights. Further analyses reveal that subnetworks found from different pre-training tend to yield diverse mask structures and perturbation sensitivities. We conclude that the core LTH observations remain generally relevant in the pre-training paradigm of computer vision, but more delicate discussions are needed in some cases. Codes and pre-trained models will be made available at: https://github. com/VITA-Group/CV_LTH_Pre-training.

show abstract

Section: Related Workmentioning

confidence: 99%

The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models

Chen¹,

Frankle²,

Chang³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Image translations as small as one pixel can result in a radically different image representation at the deepest layers of state-of-the-art CNNs [2], which means that CNNs can struggle to generalize to the wide range of translations seen in video data. In fact, small translations of input images can be effective adversarial attacks on CNNs [2,11,29]. It is also important to note that CNNs are often trained on datasets like ImageNet [10] that have demonstrable location bias: the photographed objects' locations are not equally distributed throughout the dataset, and traditional data augmentation strategies do not sufficiently address this problem [2,11].…”

Section: Related Workmentioning

confidence: 99%

An Analysis of Deep Object Detectors For Diver Detection

Langis¹,

Fulton²,

Sattar³

2020

Preprint

View full text Add to dashboard Cite

With the end goal of selecting and using diver detection models to support human-robot collaboration capabilities such as diver following, we thoroughly analyze a large set of deep neural networks for diver detection. We begin by producing a dataset of approximately 105,000 annotated images of divers sourced from videos -one of the largest and most varied diver detection datasets ever created. Using this dataset, we train a variety of state-of-the-art deep neural networks for object detection, including SSD with Mobilenet, Faster R-CNN, and YOLO. Along with these single-frame detectors, we also train networks designed for detection of objects in a video stream, using temporal information as well as single-frame image information. We evaluate these networks on typical accuracy and efficiency metrics, as well as on the temporal stability of their detections. Finally, we analyze the failures of these detectors, pointing out the most common scenarios of failure. Based on our results, we recommend SSDs or Tiny-YOLOv4 for real-time applications on robots and recommend further investigation of video object detection methods.

show abstract

“…assumption does not always hold. For instance, an autonomous car should not flip predictions for the same object between consecutive video frames due to marginal spatial shift or image noise [15], [16]. Recently, aliasing has been identified as one of the main reasons behind CNN's lack of robustness -especially against small image transformations, such as shift [2], [3].…”

Section: Introductionmentioning

confidence: 99%

Anti-aliasing Deep Image Classifiers using Novel Depth Adaptive Blurring and Activation Function

Hossain¹,

Teng²,

Sohel³

et al. 2021

Preprint

View full text Add to dashboard Cite

Deep convolutional networks are vulnerable to image translation or shift, partly due to common down-sampling layers, e.g., max-pooling and strided convolution. These operations violate the Nyquist sampling rate and cause aliasing. The textbook solution is low-pass filtering (blurring) before downsampling, which can benefit deep networks as well. Even so, nonlinearity units, such as ReLU, often re-introduce the problem, suggesting that blurring alone may not suffice. In this work, first, we analyse deep features with Fourier transform and show that Depth Adaptive Blurring is more effective, as opposed to monotonic blurring. To this end, we outline how this can replace existing down-sampling methods. Second, we introduce a novel activation function -with a built-in low pass filter, to keep the problem from reappearing. From experiments, we observe generalisation on other forms of transformations and corruptions as well, e.g., rotation, scale, and noise. We evaluate our method under three challenging settings: (1) a variety of image translations; (2) adversarial attacks -both p bounded and unbounded; and (3) data corruptions and perturbations. In each setting, our method achieves state-of-the-art results and improves clean accuracy on various benchmark datasets.

show abstract

Shift Equivariance in Object Detection

Cited by 3 publications

References 22 publications

The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models

The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models

An Analysis of Deep Object Detectors For Diver Detection

Anti-aliasing Deep Image Classifiers using Novel Depth Adaptive Blurring and Activation Function

Contact Info

Product

Resources

About