A heuristic approach to the hyperparameters in training spiking neural networks using spike-timing-dependent plasticity

Połap, Dawid; Woźniak, Marcin; Hołubowski, Waldemar; Damaševičius, Robertas

doi:10.1007/s00521-021-06824-8

Cited by 10 publications

(5 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The proposed approach involves two layers of base learners and a final meta-learner used to optimize the prediction accuracy. Other approaches involving a more accurate representation of neuronal activity use spiking neural networks [24]. To optimize the recognition rate, various heuristic algorithms including Cuckoo Search Algorithm, Grasshopper Optimization Algorithm, and Polar Bears Algorithm are used to compute the parameters of the spiking NN.…”

Section: Hybrid Methodsmentioning

confidence: 99%

Classical, Evolutionary, and Deep Learning Approaches of Automated Heart Disease Prediction: A Case Study

et al. 2023

View full text Add to dashboard Cite

Cardiovascular diseases (CVDs) are the leading cause of death globally. Detecting this kind of disease represents the principal concern of many scientists, and techniques belonging to various fields have been developed to attain accurate predictions. The aim of the paper is to investigate the potential of the classical, evolutionary, and deep learning-based methods to diagnose CVDs and to introduce a couple of complex hybrid techniques that combine hyper-parameter optimization algorithms with two of the most successful classification procedures: support vector machines (SVMs) and Long Short-Term Memory (LSTM) neural networks. The resulting algorithms were tested on two public datasets: the data recorded by the Cleveland Clinic Foundation for Heart Disease together with its extension Statlog, two of the most significant medical databases used in automated prediction. A long series of simulations were performed to assess the accuracy of the analyzed methods. In our experiments, we used F1 score and MSE (mean squared error) to compare the performance of the algorithms. The experimentally established results together with theoretical consideration prove that the proposed methods outperform both the standard ones and the considered statistical methods. We have developed improvements to the best-performing algorithms that further increase the quality of their results, being a useful tool for assisting the professionals in diagnosing CVDs in early stages.

show abstract

Section: Hybrid Methodsmentioning

confidence: 99%

Classical, Evolutionary, and Deep Learning Approaches of Automated Heart Disease Prediction: A Case Study

et al. 2023

View full text Add to dashboard Cite

show abstract

“…The use of hyperparameters in a neural network will not always match the entire dataset variable, this is still a drawback in this feature [25]. Research conducted by [26] focus on making settings in the hyperparameter to find the best settings for the entire dataset used in the study because these features are not good for all datasets used. As for the training targets and test targets, flood event data are used which are defined as floods (1) and no floods (0).…”

Section: E Data Analysis Methodsmentioning

confidence: 99%

Prediction of flood events in the city of Bandar Lampung using the artificial neural network

2023

View full text Add to dashboard Cite

The city of Bandar Lampung is currently experiencing seasonal flooding which occurs almost every year, resulting in significant losses. Floods recorded by BNPB in the last 10 years there were 16 incidents of flooding in the Bandar Lampung area. More than 14,000 people suffered, more than 500 people had to be evacuated, more than 900 houses were damaged, and 4 public facilities were damaged. To study the pattern of flood events in the past, the Artificial Neural Network Backpropagation learning method will be used which will utilize its non-linear variable learning abilities. The configuration settings for the Artificial Neural Network were carried out experimentally without any basis for assigning values, especially for the parameters of the number of hidden layers, number of neurons, and epochs used in training and variable testing. The results obtained from this study are the results of training and testing of datasets that have been carried out by ANN backpropagation are able to properly study patterns of flood events and also non-flood events in the dataset, this is evidenced by the results of high model configuration accuracy and also the results of predictive tables that able to describe actual conditions, setting the configuration model experimentally is able to produce an accuracy value of 90-100%, an average training correlation value of 0.96 and an average test correlation value of 0.89, and an average error value of 0.0089 out of 20 model configuration, and the flood prediction table are made based on the 1 best configuration with a training and testing accuracy rate of 100% with an error value of 0.00134, namely configuration model 20, the prediction table uses an average air temperature of 27˚C with 80% humidity. The prediction table is able to produce excellent flood potential results which are able to represent flood events as well as non-flood events based on the results of the dataset learning.

show abstract

“…Recent NAS procedures can be categorized broadly into four classes: Heuristics based [18], [19], Reinforcement learningbased [20], evolutionary algorithm based [21], and gradientbased [22] approaches. Heuristics based approaches are experimental approaches, and hence, require a large computation time [19]. In [19], Polap et al experimented with several heuristic algorithm to tune the hyperparameters for a specific task.…”

Section: Related Workmentioning

confidence: 99%

“…Heuristics based approaches are experimental approaches, and hence, require a large computation time [19]. In [19], Polap et al experimented with several heuristic algorithm to tune the hyperparameters for a specific task. Subramanian et al applied heuristics to find out suitable hyperparameter values related to a specific problem of identification of diseases in leaves [18].…”

Section: Related Workmentioning

confidence: 99%

Autotuned Classification Based on Knowledge Transferred from Self-supervised Models

Kishore

Mukherjee

2022

Preprint

View full text Add to dashboard Cite

With the recent advancements of deep learning-based methods in image classification, the requirement of a huge amount of training data is inevitable to avoid overfitting problems. Moreover, supervised deep learning models require labeled datasets. Preparing such a huge amount of labeled data takes a lot of human effort and time. In this scenario, self-supervised models are becoming popular because of their ability to learn even from unlabelled datasets. This paper presents a method to automatically tune the hyperparameters of a self-supervised model for image classification. The learned features are transferred from a self-supervised model to a set of Fully Connected (FC) layer, where the hyperparameters related to the architecture are tuned. A softmax layer is used following the FC layers for classification. The hyperparameters such as the number of layers, number of units in each layer, learning rate, dropout, and optimizer are automatically tuned in these FC layers, using a Bayesian optimization technique called the Tree-structured Parzen Estimator Approach (TPE) algorithm. To evaluate the performance of the proposed method, state-of-the-art self-supervised models such as Simclr and SWAV are used to extract the learned features. Experiments are carried out on CIFAR-10, CIFAR-100, and Tiny Imagenet datasets. The proposed method outperforms the state-of-the-art.

show abstract

A heuristic approach to the hyperparameters in training spiking neural networks using spike-timing-dependent plasticity

Cited by 10 publications

References 33 publications

Classical, Evolutionary, and Deep Learning Approaches of Automated Heart Disease Prediction: A Case Study

Classical, Evolutionary, and Deep Learning Approaches of Automated Heart Disease Prediction: A Case Study

Prediction of flood events in the city of Bandar Lampung using the artificial neural network

Autotuned Classification Based on Knowledge Transferred from Self-supervised Models

Contact Info

Product

Resources

About