A comparison of activation functions in artificial neural networks

Bircanoglu, Cenk; Arıca, Nafiz

doi:10.1109/siu.2018.8404724

Cited by 35 publications

(20 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Irrespective of the heterogeneous population, ANN, showed only five misclassifications in the healthy young-middle age group, two were assigned to the healthy older group and three to the patient group showing an impressive performance for this group in particular. In contrast to both SVM and RF, ANN can analyze the complex structure among variables, by using various activation functions (e.g., Tanh, Sigmoid) even though, similar to SVM the variable interactions are not visible 30,33 . Although a large data set is required to find the optimal activation function and avoid overfitting 22 , ANN has the capacity to adapt for the limited dataset with suitable activation functions which may need to be adjusted depending on the type of data 33,39 , to build a small neural network.…”

Section: Discussionmentioning

confidence: 99%

“…Nevertheless, RF disregard the intact interactions within and between trees, which might negatively impact the classification performance 32 . Although the black box problem also exists in the hidden layers of ANN 30 , the activation functions such as the tangents hyperbolic can properly analyze the complex interactions among the gait variables to improve the classification performance 33 . A recent study used deep learning to explain gait patterns based on kinematic and kinetic variables.…”

mentioning

confidence: 99%

See 1 more Smart Citation

The detection of age groups by dynamic gait outcomes using machine learning approaches

Zhou

Romijnders

Hansen

et al. 2020

Sci Rep

View full text Add to dashboard Cite

prevalence of gait impairments increases with age and is associated with mobility decline, fall risk and loss of independence. for geriatric patients, the risk of having gait disorders is even higher. consequently, gait assessment in the clinics has become increasingly important. the purpose of the present study was to classify healthy young-middle aged, older adults and geriatric patients based on dynamic gait outcomes. Classification performance of three supervised machine learning methods was compared. From trunk 3D-accelerations of 239 subjects obtained during walking, 23 dynamic gait outcomes were calculated. Kernel principal component Analysis (KpcA) was applied for dimensionality reduction of the data for Support Vector Machine (SVM) classification. Random Forest (RF) and Artificial Neural Network (ANN) were applied to the 23 gait outcomes without prior data reduction. Classification accuracy of SVM was 89%, RF accuracy was 73%, and ANN accuracy was 90%. Gait outcomes that significantly contributed to classification included: Root Mean Square (Anterior-Posterior, Vertical), cross entropy (Medio-Lateral, Vertical), Lyapunov exponent (Vertical), step regularity (Vertical) and gait speed. ANN is preferable due to the automated data reduction and significant gait outcome identification. For clinicians, these gait outcomes could be used for diagnosing subjects with mobility disabilities, fall risk and to monitor interventions. Over the last decades, medical and technical developments have extended human lifespan. However, with the increasing number of adults in society, there is a parallel increase in the number of people with serious impairments of mobility, gait, and postural control 1. Natural aging comes hand in hand with mobility decline and impairments in gait and postural control. When the level of decline in physical and cognitive functions exceeds the degree of decline expected due to the natural aging process, we speak of a geriatric condition. Typical geriatric patients are characterized by co-morbidities such as sarcopenia, cognitive impairment, osteoporosis, weight loss, and frailty 2,3. Gait disorders are common in older adults; prevalence increases with age and is associated with increased fall risk, mobility decline, and loss of independence 4. For geriatric patients, the risk of having gait disorders with an increased fall incidence is even higher 5. Consequently, objective gait assessment in the clinics has become increasingly important for the diagnosis of motor impairments and the assessment of mobility decline and fall risk 6 , as well as for the monitoring of the efficacy of interventions designed to improve mobility 7. The most often used gait parameter for disability is gait speed. After age 60, gait speed slows by 16% per decade 8. In geriatric patients, a gait speed below 1.0 m/s signifies an additional clinical or sub-clinical impairment, such as mobility decline, frailty, recurrent falling, loss of independence and institutionalization 9. Complementary to gait speed, aging impacts th...

show abstract

Section: Discussionmentioning

confidence: 99%

mentioning

confidence: 99%

The detection of age groups by dynamic gait outcomes using machine learning approaches

Zhou

Romijnders

Hansen

et al. 2020

Sci Rep

View full text Add to dashboard Cite

show abstract

“…"ReLU" was used as the active function [18]. The "batch normalization" [19] was applied to each hidden layer, and 20% of the training sets were used as the validation data sets to prevent overfitting issues. To prevent "overfitting" in the process of developing the ANN model, the "training loss" was compared against the "val loss".…”

Section: Simulationmentioning

confidence: 99%

Prediction Model for Random Variation in FinFET Induced by Line-Edge-Roughness (LER)

et al. 2021

View full text Add to dashboard Cite

As the physical size of MOSFET has been aggressively scaled-down, the impact of process-induced random variation (RV) should be considered as one of the device design considerations of MOSFET. In this work, an artificial neural network (ANN) model is developed to investigate the effect of line-edge roughness (LER)-induced random variation on the input/output transfer characteristics (e.g., off-state leakage current (Ioff), subthreshold slope (SS), saturation drain current (Id,sat), linear drain current (Id,lin), saturation threshold voltage (Vth,sat), and linear threshold voltage (Vth,lin)) of 5 nm FinFET. Hence, the prediction model was divided into two phases, i.e., “Predict Vth” and “Model Vth”. In the former, LER profiles were only used as training input features, and two threshold voltages (i.e., Vth,sat and Vth,lin) were target variables. In the latter, however, LER profiles and the two threshold voltages were used as training input features. The final prediction was then made by feeding the output of the first model to the input of the second model. The developed models were quantitatively evaluated by the Earth Mover Distance (EMD) between the target variables from the TCAD simulation tool and the predicted variables of the ANN model, and we confirm both the prediction accuracy and time-efficiency of our model.

show abstract

“…An activation function is used to determine whether the sum of the input causes activation or not. According to the researchers [14,15], there are many activation functions and amongst them, rectified linear unit (ReLU) is one of the best activation functions to carry out the DNN. Especially, Pedamonti [14] mentioned that ReLU is a better neuron replacing sigmoid function, and Cent et al [15] concluded that ReLU is the best activation function after reviewing 10 activation functions.…”

Section: Activation Functionmentioning

confidence: 99%

“…According to the researchers [14,15], there are many activation functions and amongst them, rectified linear unit (ReLU) is one of the best activation functions to carry out the DNN. Especially, Pedamonti [14] mentioned that ReLU is a better neuron replacing sigmoid function, and Cent et al [15] concluded that ReLU is the best activation function after reviewing 10 activation functions. Therefore, the widely used ReLU activation function in recent years is adopted as an activation function due to the benefits of this function like faster computation and avoiding the vanishing gradient problem [14][15][16].…”

Section: Activation Functionmentioning

confidence: 99%

Burst Pressure Prediction of API 5L X-Grade Dented Pipelines Using Deep Neural Network

Race

Oterkus

et al. 2020

JMSE

View full text Add to dashboard Cite

Mechanical damage is recognized as a problem that reduces the performance of oil and gas pipelines and has been the subject of continuous research. The artificial neural network in the spotlight recently is expected to be another solution to solve the problems relating to the pipelines. The deep neural network, which is on the basis of artificial neural network algorithm and is a method amongst various machine learning methods, is applied in this study. The applicability of machine learning techniques such as deep neural network for the prediction of burst pressure has been investigated for dented API 5L X-grade pipelines. To this end, supervised learning is employed, and the deep neural network model has four layers with three hidden layers, and the neural network uses the fully connected layer. The burst pressure computed by deep neural network model has been compared with the results of finite element analysis based parametric study, and the burst pressure calculated by the experimental results. According to the comparison results, it showed good agreement. Therefore, it is concluded that deep neural networks can be another solution for predicting the burst pressure of API 5L X-grade dented pipelines.

show abstract

A comparison of activation functions in artificial neural networks

Cited by 35 publications

References 6 publications

The detection of age groups by dynamic gait outcomes using machine learning approaches

The detection of age groups by dynamic gait outcomes using machine learning approaches

Prediction Model for Random Variation in FinFET Induced by Line-Edge-Roughness (LER)

Burst Pressure Prediction of API 5L X-Grade Dented Pipelines Using Deep Neural Network

Contact Info

Product

Resources

About