FNA++: Fast Network Adaptation via Parameter Remapping and Architecture Search

Fang, Jiemin; Sun, Yuzhu; Zhang, Qian; Peng, Kangjian; Li, Yuan; Liu, Wenyu; Wang, Xinggang

doi:10.1109/tpami.2020.3044416

Cited by 34 publications

(29 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Also, input features are downsampled aiming to eliminate spatial redundancy. Two approaches for transferring architectures in NAS are presented in [16] [17]. Last but not least, Yu et al introduce a NAS methodology tailored to face spoofing detection that considers a search space ad-hoc for the task [18].…”

Section: The Automl Special Sectionmentioning

confidence: 99%

Guest Editorial: Automated Machine Learning

Yao¹,

Tu²,

Pillay

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Section: The Automl Special Sectionmentioning

confidence: 99%

Guest Editorial: Automated Machine Learning

Yao¹,

Tu²,

Pillay

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

“…Although these lightweight networks have achieved good detection performance, this manual design cost is high and becomes difficult as the network becomes complex. Besides, there are also some recent works [36][37][38] on automatically designing network architectures. The search space of these methods is extremely large, thus one needs to train hundreds of models to distinguish good from bad ones.…”

Section: Lightweight Network Designmentioning

confidence: 99%

Learning compact ConvNets through filter pruning based on the saliency of a feature map

Liu

et al. 2021

IET Image Processing

View full text Add to dashboard Cite

With the performance increase of convolutional neural network (CNN), the disadvantages of CNN's high storage and high power consumption are followed. Among the methods mentioned in various literature, filter pruning is a crucial method for constructing lightweight networks. However, the current filter pruning method is still challenged by complicated processes and training inefficiency. This paper proposes an effective filter pruning method, which uses the saliency of the feature map (SFM), i.e. information entropy, as a theoretical guide for whether the filter is essential. The pruning principle use here is that the filter with a weak saliency feature map in the early stage will not significantly improve the final accuracy. Thus, one can efficiently prune the non-salient feature map with a smaller information entropy and the corresponding filter. Besides, an over-parameterized convolution method is employed to improve the pruned model's accuracy without increasing parameter at inference time. Experimental results show that without introducing any additional constraints, the effectiveness of this method in FLOPs and parameters reduction with similar accuracy has advanced the state-of-the-art. For example, on CIFAR-10, the pruned VGG-16 achieves only a small loss of 0.39% in Top-1 accuracy with a factor of 83.3% parameters, and 66.7% FLOPs reductions. On ImageNet-100, the pruned ResNet-50 achieves only a small accuracy degradation of 0.76% in Top-1 accuracy with a factor of 61.19% parameters, and 62.98% FLOPs reductions.

show abstract

“…Obviously, from the pixel mapping we established, we can see that the pixel region included in the dilated convolution is a subset of the pixel region of the standard convolution. Inspired by [27], we directly copy the weights of the pre-trained standard convolution to the non-zero element position of the dilated convolution, which is equivalent to the dilated convolution retaining some feature information extracted by the original convolution.…”

Section: Dilated Rate Selection For Multi-scale Extractionmentioning

confidence: 99%

Ship Detection via Dilated Rate Search and Attention-Guided Feature Representation

Zhi

Shi

et al. 2021

Remote Sensing

View full text Add to dashboard Cite

Due to the complexity of scene interference and the variability of ship scale and position, automatic ship detection in remote sensing images makes for challenging research. The existing deep networks rarely design receptive fields that fit the target scale based on training data. Moreover, most of them ignore the effective retention of position information in the feature extraction process, which reduces the contribution of features to subsequent classification. To overcome these limitations, we propose a novel ship detection framework combining the dilated rate selection and attention-guided feature representation strategies, which can efficiently detect ships of different scales under the interference of complex environments such as clouds, sea clutter and mist. Specifically, we present a dilated convolution parameter search strategy to adaptively select the dilated rate for the multi-branch extraction architecture, adaptively obtaining context information of different receptive fields without sacrificing the image resolution. Moreover, to enhance the spatial position information of the feature maps, we calculate the correlation of spatial points from the vertical and horizontal directions and embed it into the channel compression coding process, thus generating the multi-dimensional feature descriptors which are sensitive to direction and position characteristics of ships. Experimental results on the Airbus dataset demonstrate that the proposed method achieves state-of-the-art performance compared with other detection models.

show abstract

FNA++: Fast Network Adaptation via Parameter Remapping and Architecture Search

Cited by 34 publications

References 22 publications

Guest Editorial: Automated Machine Learning

Guest Editorial: Automated Machine Learning

Learning compact ConvNets through filter pruning based on the saliency of a feature map

Ship Detection via Dilated Rate Search and Attention-Guided Feature Representation

Contact Info

Product

Resources

About