Tuning hyperparameters of machine learning algorithms and deep neural networks using metaheuristics: A bioinformatics study on biomedical and biological cases

Nematzadeh, Sajjad; Kiani, Farzad; Torkamanian-Afshar, Mahsa; Aydın, Nizamettin

doi:10.1016/j.compbiolchem.2021.107619

Cited by 100 publications

(41 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the follow-up work, we will consider constructing a corpus with topical information and dig deeper into the important emotional cause of topic events. Aiming at solving the problems of feature selection and hyperparameter optimization involved in emotion cause extraction, swarm intelligence approaches will be considered [24][25][26][27][28].…”

Section: Discussionmentioning

confidence: 99%

Hierarchical Network Emotional Assistance Mechanism for Emotion Cause Extraction

Wang

Wei

Ruan

et al. 2022

Security and Communication Networks

View full text Add to dashboard Cite

The fine-grained task of emotion analysis, emotion cause extraction, is a current research hotspot. It aims to discover the underlying reasons behind the emotional expression in texts. Most of the existing work regards the task as an independent text clause classification problem, ignoring the relationship between the clauses and failing to use the indicative relationship between emotional sentences and emotional cause sentences. The existence of these problems greatly affects the accuracy of the task. In this work, an emotion cause extraction method based on a hierarchical network emotional assistance mechanism is proposed. This method uses a hierarchical network composed of bidirectional gated recurrent units, attention mechanism, and graph convolutional networks to capture clause context information, deep semantic information, and structural information between clause neighborhoods. At the same time, by enhancing the emotional information representation of the graph convolutional network nodes, the clause features of the text emotional keywords are introduced into the discovery of candidate cause sentences. Thus, a model of the deep neural network combined with the emotional assistance mechanism is established. Compared with the existing methods, the model established in this paper has better classification performance on the Chinese emotion cause dataset.

show abstract

Section: Discussionmentioning

confidence: 99%

Hierarchical Network Emotional Assistance Mechanism for Emotion Cause Extraction

Wang

Wei

Ruan

et al. 2022

Security and Communication Networks

View full text Add to dashboard Cite

show abstract

“…Moreover, as the number of hyperparameters and the range of values increase, it becomes quite difficult to manage [21]. To overcome the drawbacks of manual search, automatic search algorithms have been proposed, such as grid search [22][23][24][25]. Mainly, grid search trains machine learning models with different values of hyperparameters in the training set and compares the performance according to evaluation metrics.…”

Section: Hyperparameter Optimization For Machine Learning Modelsmentioning

confidence: 99%

“…Nematzadeh, Kiani, Torkamanian-Afshar, and Aydin (2022) [25] Proposed a method to tune hyperparameters for machine learning algorithms using Grey Wolf Optimization (GWO) and Genetic algorithm (GA) metaheuristics.…”

Section: Scheme Achievement Limitationsmentioning

confidence: 99%

Deep-Learning Model Selection and Parameter Estimation from a Wind Power Farm in Taiwan

et al. 2022

View full text Add to dashboard Cite

Deep learning networks (DLNs) use multilayer neural networks for multiclass classification that exhibit better results in wind-power forecasting applications. However, improving the training process using proper parameter hyperisations and techniques, such as regularisation and Adam-based optimisation, remains a challenge in the design of DLNs for processing time-series data. Moreover, the most appropriate parameter for the DLN model is to solve the wind-power forecasting problem by considering the excess training algorithms, such as the optimiser, activation function, batch size, and dropout. Reinforcement learning (RN) schemes constitute a smart approach to explore the proper initial parameters for the developed DLN model, considering a balance between exploration and exploitation processes. Therefore, the present study focuses on determining the proper hyperparameters for DLN models using a Q-learning scheme for four developed models. To verify the effectiveness of the developed temporal convolution network (TCN) models, experiments with five different sets of initial parameters for the TCN model were determined by the output results of Q-learning computation. The experimental results showed that the TCN accuracy for 168 h wind power prediction reached a mean absolute percentage error of 1.41%. In evaluating the effectiveness of selection of hyperparameters for the proposed model, the performance of four DLN-based prediction models for power forecasting—TCN, long short-term memory (LSTM), recurrent neural network (RNN), and gated recurrence unit (GRU) models—were compared. The overall detection accuracy of the TCN model exhibited higher prediction accuracy compared to canonical recurrent networks (i.e., the GRU, LSTM, and RNN models).

show abstract

“…It is well-known that every ML model has to be tuned for the specific dataset [2]. This also implies the no free lunch (NFL) theorem, which states that there is no universal approach, nor set of parameters' values that can render satisfying results for all practical problems.…”

Section: Introductionmentioning

confidence: 99%

Tuning Machine Learning Models Using a Group Search Firefly Algorithm for Credit Card Fraud Detection

Jovanovic¹,

Antonijevic

Stanković

et al. 2022

Mathematics

View full text Add to dashboard Cite

Recent advances in online payment technologies combined with the impact of the COVID-19 global pandemic has led to a significant escalation in the number of online transactions and credit card payments being executed every day. Naturally, there has also been an escalation in credit card frauds, which is having a significant impact on the banking institutions, corporations that issue credit cards, and finally, the vendors and merchants. Consequently, there is an urgent need to implement and establish proper mechanisms that can secure the integrity of online card transactions. The research presented in this paper proposes a hybrid machine learning and swarm metaheuristic approach to address the challenge of credit card fraud detection. The novel, enhanced firefly algorithm, named group search firefly algorithm, was devised and then used to a tune support vector machine, an extreme learning machine, and extreme gradient-boosting machine learning models. Boosted models were tested on the real-world credit card fraud detection dataset, gathered from the transactions of the European credit card users. The original dataset is highly imbalanced; to further analyze the performance of tuned machine learning models, in the second experiment performed for the purpose of this research, the dataset has been expanded by utilizing the synthetic minority over-sampling approach. The performance of the proposed group search firefly metaheuristic was compared with other recent state-of-the-art approaches. Standard machine learning performance indicators have been used for the evaluation, such as the accuracy of the classifier, recall, precision, and area under the curve. The experimental findings clearly demonstrate that the models tuned by the proposed algorithm obtained superior results in comparison to other models hybridized with competitor metaheuristics.

show abstract

Tuning hyperparameters of machine learning algorithms and deep neural networks using metaheuristics: A bioinformatics study on biomedical and biological cases

Cited by 100 publications

References 45 publications

Hierarchical Network Emotional Assistance Mechanism for Emotion Cause Extraction

Hierarchical Network Emotional Assistance Mechanism for Emotion Cause Extraction

Deep-Learning Model Selection and Parameter Estimation from a Wind Power Farm in Taiwan

Tuning Machine Learning Models Using a Group Search Firefly Algorithm for Credit Card Fraud Detection

Contact Info

Product

Resources

About