Evolving Energy Efficient Convolutional Neural Networks

Young, Steven R.; Devineni, Pravallika; Parsa, Maryam; Johnston, Jeremy; Kay, Bill; Patton, Robert M.; Schuman, Catherine D.; Rose, Derek; Potok, Thomas E.

doi:10.1109/bigdata47090.2019.9006239

Cited by 9 publications

(5 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We believe that the incredible sparsity of biological connectomes ( 3 . 2% for the C. Elegans , Cook et al (2019)) will significantly decrease the amount of energy necessary to deploy deep learning models, perhaps through the use of neuromorphic hardware (Young et al (2019); Zhu et al (2020); Schuman et al (2022)). These connectomes may lead to more robust and resilient neural systems that sidestep many of the adversarial drawbacks of highly general network structures (Guo et al (2018); Schuman et al (2020)).…”

Section: Discussion and Future Workmentioning

confidence: 99%

Biological connectomes as a representation for the architecture of artificial neural networks

Schmidgall

Schuman

Parsa

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Grand efforts in neuroscience are working toward mapping the connectomes of many new species, including the near completion of the Drosophila melanogaster. It is important to ask whether these models could benefit artificial intelligence. In this work we ask two fundamental questions: (1) where and when biological connectomes can provide use in machine learning, (2) which design principles are necessary for extracting a good representation of the connectome. Toward this end, we translate the motor circuit of the C. Elegans nematode into artificial neural networks at varying levels of biophysical realism and evaluate the outcome of training these networks on motor and non-motor behavioral tasks. We demonstrate that biophysical realism need not be upheld to attain the advantages of using biological circuits. We also establish that, even if the exact wiring diagram is not retained, the architectural statistics provide a valuable prior. Finally, we show that while the C. Elegans locomotion circuit provides a powerful inductive bias on locomotion problems, its structure may hinder performance on tasks unrelated to locomotion such as visual classification problems.

show abstract

Section: Discussion and Future Workmentioning

confidence: 99%

Biological connectomes as a representation for the architecture of artificial neural networks

Schmidgall

Schuman

Parsa

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…As such, it is important to train with those objectives in mind. In previous work, we have extended the Bayesian optimization approach [5], [9] and the fitness function used within MENNDL [50] to incorporate multiple objectives. In future work, we plan to apply this approach to the Whetstone algorithm in order to optimize networks that are both more accurate, but also more efficient.…”

Section: Discussionmentioning

confidence: 99%

Hyperparameter Optimization in Binary Communication Networks for Neuromorphic Deployment

Parsa

Schuman

Date

et al. 2020

2020 International Joint Conference on Neural Networks (IJCNN)

Self Cite

View full text Add to dashboard Cite

Training neural networks for neuromorphic deployment is non-trivial. There have been a variety of approaches proposed to adapt back-propagation or back-propagation-like algorithms appropriate for training. Considering that these networks often have very different performance characteristics than traditional neural networks, it is often unclear how to set either the network topology or the hyperparameters to achieve optimal performance. In this work, we introduce a Bayesian approach for optimizing the hyperparameters of an algorithm for training binary communication networks that can be deployed to neuromorphic hardware. We show that by optimizing the hyperparameters on this algorithm for each dataset, we can achieve improvements in accuracy over the previous state-of-theart for this algorithm on each dataset (by up to 15 percent). This jump in performance continues to emphasize the potential when converting traditional neural networks to binary communication applicable to neuromorphic hardware.

show abstract

“…Firstly, it directly relates to the model's size in storage, making it crucial for deployment in storage-constrained devices like smartphones or embedded devices [33], [45]. Secondly, a model with fewer parameters is computationally efficient in training and inference, making it more timeefficient [46]. Additionally, lightweight models are more energy-efficient, a critical factor for battery-powered devices [47].…”

Section: Datasets and Evaluation Metrics A Efficiency Evaluation Metricsmentioning

confidence: 99%

Efficient Monocular Human Pose Estimation Based on Deep Learning Methods: A Survey

Yan,

Liu,

2024

IEEE Access

View full text Add to dashboard Cite

Human pose estimation (HPE) is a crucial computer vision task with a wide range of applications in sports medicine, healthcare, virtual reality, and human-computer interaction. The demand for real-time HPE solutions necessitates the development of efficient deep-learning models that can be deployed on resource-constrained devices. While a few surveys exist in this area, none delve deeply into the critical intersection of efficiency and performance. This survey reviews the state-of-the-art efficient deep learning approaches for real-time HPE, focusing on strategies for improving efficiency without compromising accuracy. We discuss popular backbone networks for HPE, model compression techniques, network pruning and quantization, knowledge distillation, and neural architecture search methods. Furthermore, we critically analyze the existing works, highlighting their strengths, weaknesses, and applicability to different scenarios. We also present an overview of the evaluation datasets, metrics, and design for efficient HPE. Finally, we identify research gaps and challenges in the field, providing insights and recommendations for future research directions in developing efficient and scalable HPE solutions. INDEX TERMS survey, 2D human pose estimation, 3D human pose estimation, deep learning, efficiency I. INTRODUCTION A. MOTIVATION OF HUMAN POSE ESTIMATION

show abstract

Evolving Energy Efficient Convolutional Neural Networks

Cited by 9 publications

References 20 publications

Biological connectomes as a representation for the architecture of artificial neural networks

Biological connectomes as a representation for the architecture of artificial neural networks

Hyperparameter Optimization in Binary Communication Networks for Neuromorphic Deployment

Efficient Monocular Human Pose Estimation Based on Deep Learning Methods: A Survey

Contact Info

Product

Resources

About