On the compression of neural networks using <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" altimg="si174.svg" display="inline" id="d1e2062"><mml:msub><mml:mrow><mml:mi>ℓ</mml:mi></mml:mrow><mml:mrow><mml:mn>0</mml:mn></mml:mrow></mml:msub></mml:math>-norm regularization and weight pruning

de Resende Oliveira, Felipe Dennis; Batista, Eduardo Luiz Ortiz; Seara, Rui

doi:10.1016/j.neunet.2023.12.019

Neural Networks

2024

DOI: 10.1016/j.neunet.2023.12.019

|View full text |Cite

On the compression of neural networks using ℓ0-norm regularization and weight pruning

Felipe Dennis de Resende Oliveira,

Eduardo Luiz Ortiz Batista,

Rui Seara

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2024

Publication Types

Select...

Article3

Relationship

Self Cite0

Independent3

Authors

Journals

Cited by 3 publications

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Intermediate-grained kernel elements pruning with structured sparsity

Zhang,

Zhao,

Tian

et al. 2024

Neural Networks

View full text Add to dashboard Cite

Intermediate-grained kernel elements pruning with structured sparsity

Zhang,

Zhao,

Tian

et al. 2024

Neural Networks

View full text Add to dashboard Cite

The Proximal Operator of the Piece-Wise Exponential Function

Liu,

Zhou,

Lin

2024

IEEE Signal Process. Lett.

View full text Add to dashboard Cite

On choosing initial values of iteratively reweighted ℓ1 algorithms for the piece-wise exponential penalty

Lin,

Li,

Liu

2024

Anal. Appl.

View full text Add to dashboard Cite

Computing the proximal operator of the sparsity-promoting piece-wise exponential (PiE) penalty [Formula: see text] with a given shape parameter [Formula: see text], which is treated as a popular nonconvex surrogate of [Formula: see text]-norm, is fundamental in feature selection via support vector machines, image reconstruction, zero-one programming problems, compressed sensing, neural networks, etc. Due to the nonconvexity of PiE, for a long time, its proximal operator is frequently evaluated via an iteratively reweighted [Formula: see text] algorithm, which substitutes PiE with its first-order approximation, however, the obtained solutions only are the critical point. Based on the exact characterization of the proximal operator of PiE, we explore how the iteratively reweighted [Formula: see text] solution deviates from the true proximal operator in certain regions, which can be explicitly identified in terms of [Formula: see text], the initial value and the regularization parameter in the definition of the proximal operator. Moreover, the initial value can be adaptively and simply chosen to ensure that the iteratively reweighted [Formula: see text] solution belongs to the proximal operator of PiE.

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

hi@scite.ai

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

On the compression of neural networks using ℓ0-norm regularization and weight pruning

Cited by 3 publications

References 39 publications

Intermediate-grained kernel elements pruning with structured sparsity

Intermediate-grained kernel elements pruning with structured sparsity

The Proximal Operator of the Piece-Wise Exponential Function

On choosing initial values of iteratively reweighted ℓ1 algorithms for the piece-wise exponential penalty

Contact Info

Product

Resources

About