FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions

Wan, Alvin; Dai, Xiaoliang; Zhang, Peizhao; He, Zhongshi; Tian, Yuandong; Xie, Sihong; Wu, Bichen; Yu, Matthew; Xu, Tao; Chen, Kan; Vajda, Péter; Gonzalez, Joseph E.

doi:10.1109/cvpr42600.2020.01298

Cited by 249 publications

(201 citation statements)

References 10 publications

Supporting

Mentioning

183

Contrasting

Order By: Relevance

“…As a result, the size of the search space is constrained. To address this issue, Wan et al [115] propose DMaskingNAS, a memory and computationally efficient DARTS variant. DMaskingNAS employs a masking mechanism for feature map reuse.…”

Section: Nas Based On Gdmentioning

confidence: 99%

From federated learning to federated neural architecture search: a survey

Zhu

Zhang

Jin

2021

Complex Intell. Syst.

115

View full text Add to dashboard Cite

Federated learning is a recently proposed distributed machine learning paradigm for privacy preservation, which has found a wide range of applications where data privacy is of primary concern. Meanwhile, neural architecture search has become very popular in deep learning for automatically tuning the architecture and hyperparameters of deep neural networks. While both federated learning and neural architecture search are faced with many open challenges, searching for optimized neural architectures in the federated learning framework is particularly demanding. This survey paper starts with a brief introduction to federated learning, including both horizontal, vertical, and hybrid federated learning. Then neural architecture search approaches based on reinforcement learning, evolutionary algorithms and gradient-based are presented. This is followed by a description of federated neural architecture search that has recently been proposed, which is categorized into online and offline implementations, and single- and multi-objective search approaches. Finally, remaining open research questions are outlined and promising research topics are suggested.

show abstract

Section: Nas Based On Gdmentioning

confidence: 99%

From federated learning to federated neural architecture search: a survey

Zhu

Zhang

Jin

2021

Complex Intell. Syst.

115

View full text Add to dashboard Cite

show abstract

“…Many network architectures, such as the EfficientNet Tan and Le [2019], FBNetV2 Wan et al [2020], DARTS Liu et al…”

Section: Multi-objective Network Architecture Searchmentioning

confidence: 99%

EEEA-Net: An Early Exit Evolutionary Neural Architecture Search

Termritthikun

Jamtsho

Ieamsaard

et al. 2021

Engineering Applications of Artificial Intelligence

View full text Add to dashboard Cite

The goals of this research were to search for Convolutional Neural Network (CNN) architectures, suitable for an on-device processor with limited computing resources, performing at substantially lower Network Architecture Search (NAS) costs. A new algorithm entitled an Early Exit Population Initialisation (EE-PI) for Evolutionary Algorithm (EA) was developed to achieve both goals. The EE-PI reduces the total number of parameters in the search process by filtering the models with fewer parameters than the maximum threshold. It will look for a new model to replace those models with parameters more than the threshold. Thereby, reducing the number of parameters, memory usage for model storage and processing time while maintaining the same performance or accuracy. The search time was reduced to 0.52 GPU day. This is a huge and significant achievement compared to the NAS of 4 GPU days achieved using NSGA-Net, 3,150 GPU days by the AmoebaNet model, and the 2,000 GPU days by the NASNet model. As well, Early Exit Evolutionary Algorithm networks (EEEA-Nets) yield network architectures with minimal error and computational cost suitable for a given dataset as a class of network algorithms. Using EEEA-Net on CIFAR-10, CIFAR-100, and ImageNet datasets, our experiments showed that EEEA-Net achieved the lowest error rate among state-of-the-art NAS models, with 2.46% for CIFAR-10, 15.02% for CIFAR-100, and 23.8% for ImageNet dataset. Further, we implemented this image recognition architecture for other tasks, such as object detection, semantic segmentation, and keypoint detection tasks, and, in our experiments, EEEA-Net-C2 outperformed MobileNet-V3 on all of these various tasks. (The algorithm code is available at https://github.com/chakkritte/EEEA-Net).

show abstract

“…Gao et al [92] Adversarial neural architecture search for GANs AdversarialNAS 2020 CVPR Gradient based GAN Wan et al [93] Differentiable neural architecture search for spatial and channel dimensions FBNet-V2 2020 CVPR…”

Section: Gradient Based Classificationmentioning

confidence: 99%

2D and 3D Palmprint and Palm Vein Recognition Based on Neural Architecture Search

Jia

Xia

Min

et al. 2021

Int. J. Autom. Comput.

View full text Add to dashboard Cite

Palmprint recognition and palm vein recognition are two emerging biometrics technologies. In the past two decades, many traditional methods have been proposed for palmprint recognition and palm vein recognition and have achieved impressive results. In recent years, in the field of artificial intelligence, deep learning has gradually become the mainstream recognition technology because of its excellent recognition performance. Some researchers have tried to use convolutional neural networks (CNNs) for palmprint recognition and palm vein recognition. However, the architectures of these CNNs have mostly been developed manually by human experts, which is a time-consuming and error-prone process. In order to overcome some shortcomings of manually designed CNN, neural architecture search (NAS) technology has become an important research direction of deep learning. The significance of NAS is to solve the deep learning model’s parameter adjustment problem, which is a cross-study combining optimization and machine learning. NAS technology represents the future development direction of deep learning. However, up to now, NAS technology has not been well studied for palmprint recognition and palm vein recognition. In this paper, in order to investigate the problem of NAS-based 2D and 3D palmprint recognition and palm vein recognition in-depth, we conduct a performance evaluation of twenty representative NAS methods on five 2D palmprint databases, two palm vein databases, and one 3D palmprint database. Experimental results show that some NAS methods can achieve promising recognition results. Remarkably, among different evaluated NAS methods, ProxylessNAS achieves the best recognition performance.

show abstract

FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions

Cited by 249 publications

References 10 publications

From federated learning to federated neural architecture search: a survey

From federated learning to federated neural architecture search: a survey

EEEA-Net: An Early Exit Evolutionary Neural Architecture Search

2D and 3D Palmprint and Palm Vein Recognition Based on Neural Architecture Search

Contact Info

Product

Resources

About