COPS: Controlled Pruning Before Training Starts

Wimmer, Paul; Mehnert, Jens; Condurache, Alexandru Paul

doi:10.1109/ijcnn52387.2021.9533582

Search citation statements

Order By: Relevance

Paper Sections

Select...

Related Work1

Dop: Pruning Resnet-50 With Varying Levels Of Sparsity1

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2022

Publication Types

Select...

Other1

Relationship

Self Cite0

Independent1

Authors

Journals

Cited by 1 publication

(2 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most traditional PaI methods, such as LTH, dramatically raise the pruning cost since iterative pruning is needed. A distinctive branch of PaI, One-shot Network Pruning at Initialization (OPaI), solves this problem by computing the importance of each parameter in a neural network in a single step [2,5,17,24,30,33,38,39,42]. Two basic OPaI methods are SNIP [22] and GraSP [37].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

One-shot Network Pruning at Initialization with Discriminative Image Patches

Yang¹,

Jin²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

One-shot Network Pruning at Initialization (OPaI) is an effective method to decrease network pruning costs. Recently, there is a growing belief that data is unnecessary in OPaI, e.g. [12,32]. However, we obtain an opposite conclusion by ablation experiments in two representative OPaI methods, SNIP [22] and GraSP [37]. Specifically, we find that informative data is crucial to enhancing pruning performance. In this paper, we propose two novel methods, Discriminative One-shot Network Pruning (DOP) and Super Stitching, to prune the network by high-level visual discriminative image patches. Our contributions are as follows. (1) Extensive experiments reveal that OPaI is data-dependent. (2) Super Stitching performs significantly better than the original OPaI method on benchmark ImageNet, especially in a highly compressed model.

show abstract

Section: Related Workmentioning

confidence: 99%

“…The batch size is 128. The optimization is SGD with a learning rate of 0.1 at the beginning, multiplying 0.1 at epochs [30,60,90], a momentum of 0.9, and a weight decay of 0.0001.…”

Section: Dop: Pruning Resnet-50 With Varying Levels Of Sparsitymentioning

confidence: 99%