Activation functions and their characteristics in deep neural networks

Ding, Bin; Qian, Huimin; Zhou, Jun

doi:10.1109/ccdc.2018.8407425

Cited by 166 publications

(58 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The electrical resistivity results obtained in March and June, which were the respective preceding months of rainfall, were compared to analyze the electrical resistivity behavior with respect to the rainfall. When the rainfall was 5 mm (April), the electrical resistivity decreased to 34 21.18%, and 17.96% with an increase in electrode spacing. This behavior is similar to that shown in previous research findings, in which the electrical resistivity decreased with an increase in ground moisture [32].…”

Section: Distribution Of Electrical Resistivitymentioning

confidence: 97%

“…During the model building stage, the DNN, LSTM, GRU, LSTM-DNN, and GRU-DNN deep learning algorithms were selected, and the activation function, initialization method, number of hidden layers, number of nodes, optimization technique, iterative method, and sequence length were set. Six commonly used activation functions—hyperbolic tangent (tanh), sigmoid, softplus, rectified linear unit (relu), exponential linear unit (elu), and scaled exponential linear unit (selu) [ 33 , 34 ] were selected to compare the reliability of the algorithms. The initialization method was classified based on the applied activation function.…”

Section: Application Of the Deep Learning Algorithmmentioning

confidence: 99%

See 1 more Smart Citation

Discontinuity Predictions of Porosity and Hydraulic Conductivity Based on Electrical Resistivity in Slopes through Deep Learning Algorithms

Lee

Yoon

2021

Sensors

View full text Add to dashboard Cite

Electrical resistivity is used to obtain various types of information for soil strata. Hence, the prediction of electrical resistivity is helpful to predict the future behavior of soil. The objective of this study is to apply deep learning algorithms, including deep neural network (DNN), long-short term memory (LSTM), and gated recurrent unit (GRU), to determine the reliability of electrical resistivity predictions to find the discontinuity of porosity and hydraulic conductivity. New DNN-based algorithms, i.e., LSTM-DNN and GRU-DNN, are also applied in this study. The electrical resistivity values are obtained using 101 electrodes installed at 2 m intervals on a mountaintop, and a Wenner array is selected to simplify the electrode installation and measurement. A total of 1650 electrical resistivity values are obtained for one measurement considering the electrode spacing, and accumulated data measured for 15 months are used in the deep learning analysis. A constant ratio of 6:2:2 among the training, validation, and test data, respectively, is used for the measured electrical resistivity, and the hyperparameters in each algorithm are moderated to improve the reliability. Based on the deep learning model results, the distributions of porosity and hydraulic conductivity are deduced, and an average depth of 25 m is estimated for the discontinuity depth. This paper shows that the deep learning technique is well used to predict electrical resistivity, porosity, hydraulic conductivity, and discontinuity depth.

show abstract

Section: Distribution Of Electrical Resistivitymentioning

confidence: 97%

Section: Application Of the Deep Learning Algorithmmentioning

confidence: 99%

Discontinuity Predictions of Porosity and Hydraulic Conductivity Based on Electrical Resistivity in Slopes through Deep Learning Algorithms

Lee

Yoon

2021

Sensors

View full text Add to dashboard Cite

show abstract

“…One of the important characteristics of an activation function is, it should be differentiable which in turn helps in monitoring the error to backpropagate the values to update the weights. Some of the most commonly used activation functions are-sigmoid, tanh, ReLu, Leaky ReLu, Softmax [26,27].…”

Section: Neural Networkmentioning

confidence: 99%

Learning Semantic Features for Classifying Very Large Image Datasets Using Convolution Neural Network

Rao¹,

Kapanaiah²

2021

SN COMPUT. SCI.

View full text Add to dashboard Cite

Advancements in sensors and image acquisition devices lead to tremendous increase in creation of unlabeled database of images, and traditional image retrieval approaches are inefficient in retrieving semantic images. Neural network is emerging as popular method to solve most of the state-of-the-art problems in filling the gap between low level features and high level semantics. Convolution neural network (CNN) is a category of neural network which automatically extracts the important features without any human intervention, with considerably reduced set of parameters. In this paper, a model is generated using CNN (VGG-16) architecture which combines convolution and max pooling layers at different levels using effective regularization and transfer learning with data augmentation. The effectiveness of proposed model is demonstrated on benchmarking Caltech 101 and 256 datasets. The results obtained have proven the capacity of model in understanding high level semantics and outperformed several contemporary techniques.

show abstract

“…In all ANN‐based learners, stochastic gradient descent (SGD) was used as the solver that is a learning algorithm. The activation function is used to simulate the response state of the biological neuron and to obtain neuron output [38]. In all ANN‐based learners, that (hyperbolic tangent) gave the best predictive performance.…”

Section: Training Ensemble Classifiers and Base Learnersmentioning

confidence: 99%

Prediction of traumatic pathology by classifying thorax trauma using a hybrid method for emergency services

Karacı

Özkaraca

Acar

et al. 2020

IET signal process.

View full text Add to dashboard Cite

In recent years, data mining and algorithm‐based methods have been used frequently for the prediction and diagnosis of various diseases. Traumas, being one of the significant health problems in the world, are also one of the most important causes of death. This study aims to predict the presence of traumatic pathology in the lung of the patients admitted to the emergency department due to blunt thorax trauma with no X‐ray and computed tomography (CT) history by machine learning methods. The models developed in the study using the 5‐fold cross‐validation method are most accurately classified by the ensemble (voting) classifier, whether there is a pathology in X‐ray (mean accuracy = 0.82) and CT (mean accuracy = 0.83). The K‐nearest neighbourhood method classifies patients with pathology in X‐ray by 83% accuracy, while the ensemble (voting) method classifies non‐pathology patients by 94% accuracy in models. Of CT results, random forest, ensemble (voting), and ensemble (stacking) classifiers are precisely classified by 96%, while those patients with pathology are classified perspicuously by 77%. As a result, a mathematical framework using data mining methods was proposed based on estimating the X‐ray and CT results for the thorax graph scan.

show abstract

Activation functions and their characteristics in deep neural networks

Cited by 166 publications

References 10 publications

Discontinuity Predictions of Porosity and Hydraulic Conductivity Based on Electrical Resistivity in Slopes through Deep Learning Algorithms

Discontinuity Predictions of Porosity and Hydraulic Conductivity Based on Electrical Resistivity in Slopes through Deep Learning Algorithms

Learning Semantic Features for Classifying Very Large Image Datasets Using Convolution Neural Network

Prediction of traumatic pathology by classifying thorax trauma using a hybrid method for emergency services

Contact Info

Product

Resources

About