Multi-Objective Evolutionary Design of Deep Convolutional Neural Networks for Image Classification

Lu, Zhichao; Whalen, Ian; Dhebar, Yashesh; Deb, Kalyanmoy; Goodman, Erik D.; Banzhaf, Wolfgang; Boddeti, Vishnu Naresh

doi:10.48550/arxiv.1912.01369

Cited by 6 publications

(11 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Rather than generating the entire CNNs, the micro search space [45] has also been successfully employed by many recent EA-based NAS algorithms [83,84,85,86,87]. Real et al [85] propose an extension of the large-scale evolution [73], called AmoebaNet, which has achieved better results on ImageNet compared with hand-designed methods for the first time.…”

Section: Xie Et Al Proposed a Genetic Cnnmentioning

confidence: 99%

“…To reduce the computational burden for fitness evaluations, another widely adopted approaches are to train and evaluate individuals using proxy metrics [86,85,87]. The performance of the proxy models is used as the surrogate measurements to guide the evolutionary search.…”

Section: Xie Et Al Proposed a Genetic Cnnmentioning

confidence: 99%

“…Hence, the EcoNAS method is able to achieve a 400× reduced search time in comparison to AmoebaNet [85] without sacrificing the performance. Lu et al [87] empirically establish the trade-off between the correlation of proxy performance to true performance and the speed-up in estimation.…”

Section: Xie Et Al Proposed a Genetic Cnnmentioning

confidence: 99%

“…Moreover, a Bayesian network (BN) is adopted to learn the knowledge about promising architectures present in the search history and then guide the future exploitation in generating new architectures. Subsequently, Lu et al suggest NSGANet-v2 [87], an extension of NSGANet [84], where a comprehensive search space including more layer operations and one more option that controls the width of the model is introduced. Dong et al [95] present a DPP-Net on the basis of [96] that optimizes both GPU memory usage and the model performance.…”

Section: Xie Et Al Proposed a Genetic Cnnmentioning

confidence: 99%

See 3 more Smart Citations

From Federated Learning to Federated Neural Architecture Search: A Survey

Zhu

Zhang

Jin

2020

Preprint

View full text Add to dashboard Cite

Federated learning is a recently proposed distributed machine learning paradigm for privacy preservation, which has found a wide range of applications where data privacy is of primary concern. Meanwhile, neural architecture search has become very popular in deep learning for automatically tuning the architecture and hyperparameters of deep neural networks. While both federated learning and neural architecture search are faced with many open challenges, searching for optimized neural architectures in the federated learning framework is particularly demanding. This survey paper starts with a brief introduction to federated learning, including both horizontal, vertical, and hybrid federated learning. Then, neural architecture search approaches based on reinforcement learning, evolutionary algorithms and gradient-based are presented. This is followed by a description of federated neural architecture search that has recently been proposed, which is categorized into online and offline implementations, and single-and multi-objective search approaches. Finally, remaining open research questions are outlined and promising research topics are suggested.

show abstract

Section: Xie Et Al Proposed a Genetic Cnnmentioning

confidence: 99%

Section: Xie Et Al Proposed a Genetic Cnnmentioning

confidence: 99%

Section: Xie Et Al Proposed a Genetic Cnnmentioning

confidence: 99%

Section: Xie Et Al Proposed a Genetic Cnnmentioning

confidence: 99%

See 2 more Smart Citations

From Federated Learning to Federated Neural Architecture Search: A Survey

Zhu

Zhang

Jin

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…However, most of them treat it as a single-objective optimization problem, which is not well suited for solving the trade-off problem. Inspired by the work in [29], [30], [31], we adopt a multi-objective approach based NAS to find the best architecture for the trade-off solutions between the adversarial accuracy, the clean accuracy and the mode size. In our work, we mainly address the trade-off problem based on the ShuffleNetV2 architecture [32], Xception block [33], SE layer [34], Non-Local block [35], and their variants.…”

Section: Introductionmentioning

confidence: 99%

Tiny Adversarial Mulit-Objective Oneshot Neural Architecture Search

Xie

Wang²,

et al. 2021

Preprint

View full text Add to dashboard Cite

Lights and Shadows in Evolutionary Deep Learning: Taxonomy, Critical Methodological Analysis, Cases of Study, Learned Lessons, Recommendations and Challenges

Martinez¹,

Ser²,

Villar-Rodríguez³

et al. 2020

Preprint

View full text Add to dashboard Cite

Much has been said about the fusion of bio-inspired optimization algorithms and Deep Learning models for several purposes: from the discovery of network topologies and hyperparametric configurations with improved performance for a given task, to the optimization of the models parameters as a replacement for gradient-based solvers. Indeed, the literature is rich in proposals showcasing the application of assorted nature-inspired approaches for these tasks. In this work we comprehensively review and critically examine contributions made so far based on three axes, each addressing a fundamental question in this research avenue: a) optimization and taxonomy (Why?), including a historical perspective, definitions of optimization problems in Deep Learning, and a taxonomy associated with an in-depth analysis of the literature, b) critical methodological analysis (How?), which together with two case studies, allows us to address learned lessons and recommendations for good practices following the analysis of the literature, and c) challenges and new directions of research (What can be done, and what for?). In summary, three axes -optimization and taxonomy, critical analysis, and challenges -which outline a complete vision of a merger of two technologies drawing up an exciting future for this area of fusion research.

show abstract

Multi-Objective Evolutionary Design of Deep Convolutional Neural Networks for Image Classification

Cited by 6 publications

References 52 publications

From Federated Learning to Federated Neural Architecture Search: A Survey

From Federated Learning to Federated Neural Architecture Search: A Survey

Tiny Adversarial Mulit-Objective Oneshot Neural Architecture Search

Lights and Shadows in Evolutionary Deep Learning: Taxonomy, Critical Methodological Analysis, Cases of Study, Learned Lessons, Recommendations and Challenges

Contact Info

Product

Resources

About