2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2021
DOI: 10.1109/cvpr46437.2021.01215
|View full text |Cite
|
Sign up to set email alerts
|

Cross-Iteration Batch Normalization

Abstract: A well-known issue of Batch Normalization is its significantly reduced effectiveness in the case of small mini-batch sizes. When a mini-batch contains few examples, the statistics upon which the normalization is defined cannot be reliably estimated from it during a training iteration. To address this problem, we present Cross-Iteration Batch Normalization (CBN), in which examples from multiple recent iterations are jointly utilized to enhance estimation quality. A challenge of computing statistics over multipl… Show more

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
1
1
1
1

Citation Types

0
15
0

Year Published

2021
2021
2023
2023

Publication Types

Select...
7
2
1

Relationship

0
10

Authors

Journals

citations
Cited by 64 publications
(16 citation statements)
references
References 24 publications
0
15
0
Order By: Relevance
“…YOLO v4 [48] has put to the test a large variety of strategies that are supposed to enhance accuracy of a CNN. Finally, it combines techniques such as Weighted-Residual-Connections [30], Cross-Stage-Partial-Connections [49], Cross mini-Batch Normalization [50], Self-adversarial-training [51], Mish activation [52], Mosaic data augmentation, DropBlock regularization [53], and CIoU loss [54] to achieve optimal object detection speed and accuracy. In [55], to construct a lightweight underwater object detector, YOLO v4 is combined with a multi-scale attentional feature fusion module.…”
Section: One-stage Detectorsmentioning
confidence: 99%
“…YOLO v4 [48] has put to the test a large variety of strategies that are supposed to enhance accuracy of a CNN. Finally, it combines techniques such as Weighted-Residual-Connections [30], Cross-Stage-Partial-Connections [49], Cross mini-Batch Normalization [50], Self-adversarial-training [51], Mish activation [52], Mosaic data augmentation, DropBlock regularization [53], and CIoU loss [54] to achieve optimal object detection speed and accuracy. In [55], to construct a lightweight underwater object detector, YOLO v4 is combined with a multi-scale attentional feature fusion module.…”
Section: One-stage Detectorsmentioning
confidence: 99%
“…CSP implementation reduces compute by as much as 20% and outperforms other state-of-the-art backbone architectures [8]. c. Cross Mini-Batch Normalization overcomes the problem where the statistics generated when normalization is defined cannot be estimated properly [9]. d. Self-adversarial Training is a new data augmentation technique that operates in 2 stages forward and backward on the network [10].…”
Section: You Only Look Once (Yolo)mentioning
confidence: 99%
“…Bochkovskiy et al also a classified bag of specials as a plugin module and post-processing method that only increases the inference cost by a small amount but can significantly improve accuracy; this includes enlarging the receptive field, an attention module, feature integration, and post-processing. Within the bag of freebies and bag of specials that were evaluated, Bochkovskiy et al introduced 4 modifications: SPP [19], a Spatial Attention Module (SAM) [23], PAN [20], and cross-iteration batch normalization (CBN) [24].…”
Section: Mosquito Surveillance Algorithmmentioning
confidence: 99%