Improving Convolutional Neural Network Using Pseudo Derivative ReLU

Hu, Zheng; Li, Yongping; Yang, Zhiyong

doi:10.1109/icsai.2018.8599372

Cited by 21 publications

(8 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another advantage of ReLU is that it is easy to compute as the output equals the input if the input is non-negative; otherwise, it equals 0 . This ability can alleviate the gradient vanishing and exploding problems that usually occur with the sigmoid or tanh activation functions [40]. Various optimization algorithms, namely, Adam (Adaptive Moment Estimation), SGD (Stochastic Gradient Descent), and RMSprop were employed during the optimization of the model.…”

Section: Proposed Modelmentioning

confidence: 99%

A transfer learning-based deep learning approach for automated COVID-19 diagnosis with audio data

Akgün¹,

Kabakuş²,

Şentürk³

et al. 2021

Turk J Elec Eng & Comp Sci

View full text Add to dashboard Cite

The COVID-19 pandemic has caused millions of deaths and changed daily life globally. Countries have declared a half or full lockdown to prevent the spread of COVID-19. According to medical doctors, as many people as possible should be tested to identify their status, and corresponding actions then should be taken for COVID-19 positive cases. Despite the clear necessity of these medical tests, many countries are still struggling to acquire them. This fact clearly indicates the necessity of a large-scale, cheap, fast, and accurate alternative pre-screening tool that can be used for the diagnosis of COVID-19 while waiting for the medical tests. To this end, a novel end-to-end transfer learning-based deep learning approach that uses only a given cough sound for the diagnosis of COVID-19 was proposed in this study.The proposed models employed various pre-trained deep neural networks, namely, VGG19, ResNet50V2, DenseNet121, and MobileNet, via the transfer-learning technique. Then, these models were evaluated on a gold standard dataset, namely, Cambridge Data. According to the experimental result, the proposed model, which employed the MobileNet via the transfer-learning technique, provided the best accuracy, 86.42 %, and outperformed the state-of-the-art. Thus, the proposed model has the potential to provide automated COVID-19 diagnosis in an easily applicable and fast yet accurate way.

show abstract

Section: Proposed Modelmentioning

confidence: 99%

A transfer learning-based deep learning approach for automated COVID-19 diagnosis with audio data

Akgün¹,

Kabakuş²,

Şentürk³

et al. 2021

Turk J Elec Eng & Comp Sci

View full text Add to dashboard Cite

show abstract

“…The ReLU and the Softsign functions are applied. The ReLU function performs a threshold operation to set any input less than zero to zero [29]. The Softsign function has the flatr curve and slow decreasing derivatives for more efficient learning [30].…”

Section: Activation Layermentioning

confidence: 99%

Dual-Input and Multi-Channel Convolutional Neural Network Model for Vehicle Speed Prediction

Xing

Liang

Guo

et al. 2021

Sensors

View full text Add to dashboard Cite

With the development of technology, speed prediction has become an important part of intelligent vehicle control strategies. However, the time-varying and nonlinear nature of vehicle speed increases the complexity and difficulty of prediction. Therefore, a CNN-based neural network architecture with two channel input (DICNN) is proposed in this paper. With two inputs and four channels, DICNN can predict the speed changes in the next 5 s by extracting the temporal information of 10 vehicle signals and the driver’s intention. The prediction performances of DICNN are firstly examined. The best RMSE, MAE, ME and R2 are obtained compared with a Markov chain combined with Monte Carlo (MCMC) simulation, a support vector machine (SVM) and a single input CNN (SICNN). Secondly, equivalent fuel consumption minimization strategies (ECMS) combining different vehicle speed prediction methods are constructed. After verification by simulation, the equivalent fuel consumption of the simulation increases by only 4.89% compared with dynamic-programming-based energy management strategy and decreased by 5.40% compared with the speed prediction method with low accuracy.

show abstract

“…Besides, they have an acceptable degree of smoothness and are easily differentiated, 44 unlike a ReLU function, which has a differentiation problem that can lead to a dying ReLU problem. 45 The interested reader is referred to 32 for a detailed description of DNN and autoencoder algorithms.…”

Section: Fkm Of An Inter-module: Deep Neural Network-based Solutionmentioning

confidence: 99%

Forward Kinematic Modeling of Conical-Shaped Continuum Manipulators

Boutchouang¹,

Melingui²,

Ahanda³

et al. 2021

Robotica

View full text Add to dashboard Cite

SUMMARY Forward kinematics is essential in robot control. Its resolution remains a challenge for continuum manipulators because of their inherent flexibility. Learning-based approaches allow obtaining accurate models. However, they suffer from the explosion of the learning database that wears down the manipulator during data collection. This paper proposes an approach that combines the model and learning-based approaches. The learning database is derived from analytical equations to prevent the robot from operating for long periods. The database obtained is handled using Deep Neural Networks (DNNs). The Compact Bionic Handling robot serves as an experimental platform. The comparison with existing approaches gives satisfaction.

show abstract

Improving Convolutional Neural Network Using Pseudo Derivative ReLU

Cited by 21 publications

References 4 publications

A transfer learning-based deep learning approach for automated COVID-19 diagnosis with audio data

A transfer learning-based deep learning approach for automated COVID-19 diagnosis with audio data

Dual-Input and Multi-Channel Convolutional Neural Network Model for Vehicle Speed Prediction

Forward Kinematic Modeling of Conical-Shaped Continuum Manipulators

Contact Info

Product

Resources

About