Single-level Optimization For Differential Architecture Search

Hou, Pengfei; Jin, Ying

doi:10.48550/arxiv.2012.11337

Cited by 2 publications

(1 citation statement)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It first optimizes the network weights w, then finds α that minimizes the validation loss L val . The other is single-level optimization [2,34,37], which regards the optimization of w and α as independent processes. It can be described as:…”

Section: Network Architecture Searchmentioning

confidence: 99%

Efficient Re-parameterization Operations Search for Easy-to-Deploy Network Based on Directional Evolutionary Strategy

Wang

Rong

et al. 2023

Neural Process Lett

View full text Add to dashboard Cite

Structural re-parameterization (Rep) methods has achieved significant performance improvement on traditional convolutional network. Most current Rep methods rely on prior knowledge to select the reparameterization operations. However, the performance of architecture is limited by the type of operations and prior knowledge. To break this restriction, in this work, an improved re-parameterization search space is designed, which including more type of re-parameterization operations. Concretely, the performance of convolutional networks can be further improved by the search space. To effectively explore this search space, an automatic re-parameterization enhancement strategy is designed based on neural architecture search (NAS), which can search a excellent re-parameterization architecture. Besides, we visualize the output features of the architecture to analyze the reasons for the formation of the re-parameterization architecture. On public datasets, we achieve better results. Under the same training conditions as ResNet, we improve the accuracy of ResNet-50 by 1.82% on ImageNet-1k.

show abstract

Section: Network Architecture Searchmentioning

confidence: 99%

Efficient Re-parameterization Operations Search for Easy-to-Deploy Network Based on Directional Evolutionary Strategy

Wang

Rong

et al. 2023

Neural Process Lett

View full text Add to dashboard Cite

show abstract

Efficient Re-parameterization Operations Search for Easy-to-Deploy Network Based on Directional Evolutionary Strategy

Wang

Rong

et al. 2022

Preprint

View full text Add to dashboard Cite

Structural re-parameterization (Rep) methods has achieved signifificant performance improvement on traditional convolutional network. Most current Rep methods rely on prior knowledge to select the re-parameterization operations. However, the performance of architecture is limited by the type of operations and prior knowledge. To break this restriction, in this work, an improved re-parameterization search space is designed, which including more type of re-parameterization operations. Concretely, the performance of convolutional networks can be further improved by the search space. To effectively explore this search space, an automatic re-parameterization enhancement strategy is designed based on neural architecture search (NAS), which can search a excellent re-parameterization architecture. Besides, we visualize the output features of the architecture to analyze the reasons for the formation of the re-parameterization architecture. On public datasets, we achieve better results. Under the same training conditions as ResNet, we improve the accuracy of ResNet-50 by 1.82% on ImageNet-1k.

show abstract

Single-level Optimization For Differential Architecture Search

Cited by 2 publications

References 17 publications

Efficient Re-parameterization Operations Search for Easy-to-Deploy Network Based on Directional Evolutionary Strategy

Efficient Re-parameterization Operations Search for Easy-to-Deploy Network Based on Directional Evolutionary Strategy

Efficient Re-parameterization Operations Search for Easy-to-Deploy Network Based on Directional Evolutionary Strategy

Contact Info

Product

Resources

About