An Efficient Hardware Architecture for Multilayer Spiking Neural Networks

Luo, Yuling; Wan, Lei; Liu, Junxiu; Zhang, Jinlei; Cao, Yi

doi:10.1007/978-3-319-70136-3_83

Cited by 10 publications

(15 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another is mapping the search space into a low-dimensional one. For examples, using auto-encoder [25] or sparse coding [40]. However, these types of methods do not consider special properties of the search problem, e.g., the graph structure of the supernet.…”

Section: Discussionmentioning

confidence: 99%

ColdNAS: Search to Modulate for User Cold-Start Recommendation

Wang

Jing

et al. 2023

Proceedings of the ACM Web Conference 2023

View full text Add to dashboard Cite

Making personalized recommendation for cold-start users, who only have a few interaction histories, is a challenging problem in recommendation systems. Recent works leverage hypernetworks to directly map user interaction histories to user-specific parameters, which are then used to modulate predictor by feature-wise linear modulation function. These works obtain the state-of-the-art performance. However, the physical meaning of scaling and shifting in recommendation data is unclear. Instead of using a fixed modulation function and deciding modulation position by expertise, we propose a modulation framework called ColdNAS for user cold-start problem, where we look for proper modulation structure, including function and position, via neural architecture search. We design a search space which covers broad models and theoretically prove that this search space can be transformed to a much smaller space, enabling an efficient and robust one-shot search algorithm. Extensive experimental results on benchmark datasets show that ColdNAS consistently performs the best. We observe that different modulation functions lead to the best performance on different datasets, which validates the necessity of designing a searching-based method. Codes are available at https://github.com/LARS-research/ColdNAS.

show abstract

Section: Discussionmentioning

confidence: 99%

ColdNAS: Search to Modulate for User Cold-Start Recommendation

Wang

Jing

et al. 2023

Proceedings of the ACM Web Conference 2023

View full text Add to dashboard Cite

show abstract

Section: Introductionmentioning

confidence: 99%

“…On the other hand, evaluating all candidate architectures in the extremely large NAS search space is nearly impossible. To overcome these challenges, various surrogate models 13,14 and search algorithms are proposed in NAS to reduce the computational resources for evaluating candidate architectures, and improve the efficiency of search algorithm.Typically, various predictors 15,16 are developed to estimate the performance of candidate architectures. Train-free predictors 17,18 offer the advantage of providing architecture performance without requiring any training samples and have demonstrated high efficiency and effectiveness, but their performance are usually not good enough in practice.…”

mentioning

confidence: 99%

Ranking‐based architecture generation for surrogate‐assisted neural architecture search

Xiao,

Wang

2024

Concurrency and Computation

View full text Add to dashboard Cite

Architectures generation optimization has been received a lot of attention in neural architecture search (NAS) since its efficiency in generating architecture. By learning the architecture representation through unsupervised learning and constructing a latent space, the prediction process of predictors is simplified, leading to improved efficiency in architecture search. However, searching for architectures with top performance in complex and large NAS search spaces remains challenging. In this paper, an approach that combined a ranker and generative model is proposed to address this challenge through regularizing the latent space and identifying architectures with top rankings. We introduce the ranking error to gradually regulate the training of the generative model, making it easier to identify architecture representations in the latent space. Additionally, a surrogate‐assisted evolutionary search method that utilized neural network Bayesian optimization is proposed to efficiently explore promising architectures in the latent space. We demonstrate the benefits of our approach in optimizing architectures with top rankings, and our method outperforms state‐of‐the‐art techniques on various NAS benchmarks. The code is available at https://github.com/outofstyle/RAGS‐NAS.

show abstract

“…The concept of neural architecture search has emerged in response to the difficulties in designing deep learning models and time-consuming tuning of parameters [12]. Genetic algorithm [13], reinforcement learning [14] and Bayesian optimization algorithm [15] have been applied to the network architecture search process, and these methods have solved the problem of network architecture design to a certain extent, making it possible for models to automatically obtain the appropriate network architecture based on data information.…”

Section: Introductionmentioning

confidence: 99%

A gradient-based automatic optimization CNN framework for EEG state recognition

et al. 2022

View full text Add to dashboard Cite

The Electroencephalogram (EEG) signal, as a data carrier that can contain a large amount of information about the human brain in different states, is one of the most widely used metrics for assessing human psychophysiological states. Among a variety of analysis methods, deep learning, especially convolutional neural network (CNN), has achieved remarkable results in recent years as a method to effectively extract features from EEG signals. Although deep learning has the advantages of automatic feature extraction and effective classification, it also faces difficulties in network structure design and requires an army of prior knowledge. Automating the design of these hyperparameters can therefore save experts' time and manpower. Neural architecture search techniques have thus emerged. In this paper, based on an existing gradient-based NAS algorithm, PC-DARTS, with targeted improvements and optimizations for the characteristics of EEG signals. Specifically, we establish the model architecture step by step based on the manually designed deep learning models for EEG discrimination by retaining the framework of the search algorithm and performing targeted optimization of the model search space. Corresponding features are extracted separately according to the frequency domain, time domain characteristics of the EEG signal and the spatial position of the EEG electrode. The architecture was applied to EEG-based emotion recognition and driver drowsiness assessment tasks. The results illustrate that compared with the existing methods, the model architecture obtained in this paper can achieve competitive overall accuracy and better standard deviation in both tasks. Therefore, this approach is an effective migration of NAS technology into the field of EEG analysis and has great potential to provide high-performance results for other types of classification and prediction tasks. This can effectively reduce the time cost for researchers and facilitate the application of CNN in more areas.

show abstract

An Efficient Hardware Architecture for Multilayer Spiking Neural Networks

Cited by 10 publications

References 12 publications

ColdNAS: Search to Modulate for User Cold-Start Recommendation

ColdNAS: Search to Modulate for User Cold-Start Recommendation

Ranking‐based architecture generation for surrogate‐assisted neural architecture search

A gradient-based automatic optimization CNN framework for EEG state recognition

Contact Info

Product

Resources

About