Improving deep neural networks using softplus units

Zheng, Hao; Yang, Zhanlei; Liu, Wenju; Liang, Jizhong; Li, Yanpeng

doi:10.1109/ijcnn.2015.7280459

Cited by 146 publications

(86 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nwankpa et al (2018) highlighted the existing activation functions used in deep learning applications and summarizes the work done in the use of activation functions for deep learning applications. Zheng et al (2015) suggests that compared to ReLU, the smoothening and non-zero properties of the gradient make the soft-plus based Neural Networks perform better. Further (Bengio and Grandvalet 2004) studied the K-fold cross-validation method used to evaluate the proposed models on a limited data sample.…”

Section: Related Workmentioning

confidence: 99%

RETRACTED ARTICLE: Deep CNN framework for retinal disease diagnosis using optical coherence tomography images

Rajagopalan

Venkateswaran²,

Vinjimoor

et al. 2020

J Ambient Intell Human Comput

View full text Add to dashboard Cite

Accurate and robust diagnosis of retinal diseases using OCT imaging is considered an essential part for clinical utility. We propose a deep learning based, a fully automated diagnosis system for detecting retinal disorders namely, Drusen macular degeneration (DMD) and diabetic macular edema (DME) using optical coherence tomography (OCT) Images. If it is not diagnosed and treated, these degenerative abnormalities may result in moderate to severe vision loss. Early detection of these diseases reduces the risk of further complications and expedites the treatment process. We propose a deep convolutional neural network (CNN) framework for the diagnosis and classification into Normal, DMD and DME effectively. First, despeckling of the input OCT images is performed using the Kuan filter to remove inherent speckle noise. Further, the CNN network is tuned using hyperparameter optimization techniques. Additionally, K-fold validation is performed to ensure complete usage of the dataset. We evaluate the proposed model with number of performance metrics using Mendeley database consisting of labelled OCT images. The resulting classification accuracy of the proposed model is 95.7%. Further, an authoritative study is performed between the pre-trained models and proposed framework using the acquired performance metrics to demonstrate the efficacy of our model.

show abstract

Section: Related Workmentioning

confidence: 99%

RETRACTED ARTICLE: Deep CNN framework for retinal disease diagnosis using optical coherence tomography images

Rajagopalan

Venkateswaran²,

Vinjimoor

et al. 2020

J Ambient Intell Human Comput

View full text Add to dashboard Cite

show abstract

“…Since the state cost C s (s t ) is non-decreasing in s t,i and the condition does not improve (at least without maintenance), q(s t,i ) should also be nondecreasing. Considering these properties, we utilize the following parameterization for q, which is an extension of the softplus (smoothed ReLU) function [35].…”

Section: Modeling Qi: the Maintenance Priority Of I-th Targetmentioning

confidence: 99%

Combinatorial Q-Learning for Condition-Based Infrastructure Maintenance

Tanimoto

2021

IEEE Access

View full text Add to dashboard Cite

Infrastructure maintenance planning is a large-scale optimization problem of planning when and on which components to carry out maintenance so as to keep the whole infrastructure in good condition with minimal maintenance cost. Recent advances in condition monitoring techniques have enabled timely maintenance in response to the condition of each part regardless of age. In addition to the condition, the spatial structure is also important for cost-efficiency in infrastructure maintenance since traveling costs and/or setup costs can be saved by simultaneous maintenance of neighboring components, which is called economic dependency. This optimization problem naively has a high computational complexity of O(2 nH ), where n is the number of components and H is the planning horizon, and the predictive modeling of degradation is also a big issue. To solve this problem efficiently at scale, our proposed method utilizes two kinds of dynamic programming for temporal and spatial scalability and consequently enjoys O(n) complexity at each time step. For temporal scalability, we utilize a direct modeling approach for the action value of maintenance instead of modeling degradation, namely, Q-learning. For spatial scalability, we exploit locality in economic dependency by means of a reasonable approximation of the Q-function. A typical baseline approach is to divide the whole infrastructure into fixed groups of neighboring components beforehand and determine if maintenance should be performed for all the components in each group at each time step. In contrast, our scalable method enables fully combinatorial optimization for each component at each time step. We demonstrate the advantage of our method in a simulated environment, and the resulting maintenance history intuitively illustrates the benefit of our dynamic grouping approach. We also show that our method has a kind of interpretability in the optimization at each time step.

show abstract

“…The ReLU activation function family is mainly used for classification and reinforcement learning (RL) problems. The identity, LeakyReLU [30], Elu [31], and Softplus [32] functions are included in this family. The identity function y = x is typically used for the output layer of regression.…”

Section: Activation Functionsmentioning

confidence: 99%

Applications of Machine Learning

Rudström¹

2020

Algorithms for Intelligent Systems

View full text Add to dashboard Cite

In this thesis, many machine learning algorithms were applied to electrocardiogram (ECG), spectral analysis, and Field Programmable Gate Arrays (FPGAs). In ECG, QRS complexes are useful for measuring the heart rate and for the segmentation of ECG signals. QRS complexes were detected using WaveletCNN Autoencoder filters and ConvLSTM detectors. The WaveletCNN Autoencoders filters the ECG signals using the wavelet filters, while the ConvLSTM detects the spatial temporal patterns of the QRS complexes. For the spectral analysis topic, the detection of chemical compounds using spectral analysis is useful for identifying unknown substances. However, spectral analysis algorithms require vast amounts of data. To solve this problem, B-spline neural networks were developed for the generation of infrared and ultraviolet/visible spectras. This allowed for the generation of large training datasets from a few experimental measurements. Graphical Processing Units (GPUs) are good for training and testing neural networks. However, using multiple GPUs together is hard because PCIe bus is not suited for scattering operations and reduce operations. FPGAs are more flexible as they can be arranged in a mesh or toroid or hypercube configuration on the PCB. These configurations provide higher data throughput and results in faster computations. A general neural network framework was written in VHDL for Xilinx FPGAs. It allows for any neural network to be trained or tested on FPGAs.

show abstract

Improving deep neural networks using softplus units

Cited by 146 publications

References 7 publications

RETRACTED ARTICLE: Deep CNN framework for retinal disease diagnosis using optical coherence tomography images

RETRACTED ARTICLE: Deep CNN framework for retinal disease diagnosis using optical coherence tomography images

Combinatorial Q-Learning for Condition-Based Infrastructure Maintenance

Applications of Machine Learning

Contact Info

Product

Resources

About