Softprop: softmax neural network backpropagation leaming

Rimer, M.E.; Martinez, Tony

doi:10.1109/ijcnn.2004.1380066

Cited by 6 publications

(6 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The class scores are computed in the fully connected layer. After that, the output of the softmax layer is an N -dimensional vector ( Rimer and Martinez, 2004 ), corresponding to the number of classes desired, and N is set to two classes (normal and pathological fetuses). In this work, the cross-entropy is adopted as the loss function in the softmax classification layer.…”

Section: Methodsmentioning

confidence: 99%

Computer-Aided Diagnosis System of Fetal Hypoxia Incorporating Recurrence Plot With Convolutional Neural Network

et al. 2019

View full text Add to dashboard Cite

Background: Electronic fetal monitoring (EFM) is widely applied as a routine diagnostic tool by clinicians using fetal heart rate (FHR) signals to prevent fetal hypoxia. However, visual interpretation of the FHR usually leads to significant inter-observer and intra-observer variability, and false positives become the main cause of unnecessary cesarean sections.Goal: The main aim of this study was to ensure a novel, consistent, robust, and effective model for fetal hypoxia detection.Methods: In this work, we proposed a novel computer-aided diagnosis (CAD) system integrated with an advanced deep learning (DL) algorithm. For a 1-dimensional preprocessed FHR signal, the 2-dimensional image was transformed using recurrence plot (RP), which is considered to greatly capture the non-linear characteristics. The ultimate image dataset was enriched by changing several parameters of the RP and was then used to feed the convolutional neural network (CNN). Compared to conventional machine learning (ML) methods, a CNN can self-learn useful features from the input data and does not perform complex manual feature engineering (i.e., feature extraction and selection).Results: Finally, according to the optimization experiment, the CNN model obtained the average performance using optimal configuration across 10-fold: accuracy = 98.69%, sensitivity = 99.29%, specificity = 98.10%, and area under the curve = 98.70%.Conclusion: To the best of our knowledge, this approached achieved better classification performance in predicting fetal hypoxia using FHR signals compared to the other state-of-the-art works.Significance: In summary, the satisfied result proved the effectiveness of our proposed CAD system for assisting obstetricians making objective and accurate medical decisions based on RP and powerful CNN algorithm.

show abstract

Section: Methodsmentioning

confidence: 99%

Computer-Aided Diagnosis System of Fetal Hypoxia Incorporating Recurrence Plot With Convolutional Neural Network

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Dynamically updating the value of the error margin as training progresses is a straightforward extension to be evaluated. Softprop, a learning approach combining CB1 and SSE optimization during training by means of the error margin, has shown improvement over CB1 in a preliminary study (Rimer & Martinez, 2004) and a thorough analysis will be presented in future work. Using a value for the error margin local to each training instance and intelligently updating these values as training progresses also shows promise.…”

Section: Future Workmentioning

confidence: 97%

“…The value of µ can also be decreased, and remain negative as training is concluded to account for noisy outliers. A preliminary analysis of updating µ during training has shown promise (Rimer & Martinez, 2004).…”

Section: Increasing the Margin With Cb Trainingmentioning

confidence: 99%

Classification-based objective functions

Rimer

Martinez

2006

Mach Learn

View full text Add to dashboard Cite

Backpropagation, similar to most learning algorithms that can form complex decision surfaces, is prone to overfitting. This work presents classification-based objective functions, an approach to training artificial neural networks on classification problems. Classification-based learning attempts to guide the network directly to correct pattern classification rather than using common error minimization heuristics, such as sum-squared error (SSE) and cross-entropy (CE), that do not explicitly minimize classification error. CB1 is presented here as a novel objective function for learning classification problems. It seeks to directly minimize classification error by backpropagating error only on misclassified patterns from culprit output nodes. CB1 discourages weight saturation and overfitting and achieves higher accuracy on classification problems than optimizing SSE or CE. Experiments on a large OCR data set have shown CB1 to significantly increase generalization accuracy over SSE or CE optimization, from 97.86% and 98.10%, respectively, to 99.11%. Comparable results are achieved over several data sets from the UC Irvine Machine Learning Database Repository, with an average increase in accuracy from 90.7% and 91.3% using optimized SSE and CE networks, respectively, to 92.1% for CB1. Analysis indicates that CB1 performs a fundamentally different search of the feature space than optimizing SSE or CE and produces significantly different solutions.

show abstract

“…Prior work has shown [8][9][10] that methods of calculating softer values for each training pattern based on the network's output vector improve generalization and reduce variance on classification problems over a corpus of benchmark learning problems. One of these, called lazy training or CB1, focuses on classification accuracy backpropagates an error signal through the network only when a pattern is misclassified.…”

Section: Motivation For Cb3mentioning

confidence: 99%

“…Classification-based (CB) error functions [9,10] are a relatively new method of training multi-layer perceptrons. The CB functions heuristically seek to directly minimize classification error by backpropagating network error only on misclassified patterns.…”

Section: Introductionmentioning

confidence: 99%

CB3: An Adaptive Error Function for Backpropagation Training

Rimer

Martinez

2006

Neural Process Lett

View full text Add to dashboard Cite

Effective backpropagation training of multi-layer perceptrons depends on the incorporation of an appropriate error or objective function. Classification-Based (CB) error functions are heuristic approaches that attempt to guide the network directly to correct pattern classification rather than using common error minimization heuristics, such as Sum-Squared Error (SSE) and Cross-Entropy (CE), which do not explicitly minimize classification error. This work presents CB3, a novel CB approach that learns the error function to be used while training. This is accomplished by learning pattern confidence margins during training, which are used to dynamically set output target values for each training pattern. On 11 applications, CB3 significantly outperforms previous CB error functions, and also reduces average test error over conventional error metrics using 0-1 targets without weight decay by 1.8%, and by 1.3% over metrics with weight decay. The CB3 also exhibits lower model variance and tighter mean confidence interval.

show abstract

Softprop: softmax neural network backpropagation leaming

Cited by 6 publications

References 14 publications

Computer-Aided Diagnosis System of Fetal Hypoxia Incorporating Recurrence Plot With Convolutional Neural Network

Computer-Aided Diagnosis System of Fetal Hypoxia Incorporating Recurrence Plot With Convolutional Neural Network

Classification-based objective functions

CB3: An Adaptive Error Function for Backpropagation Training

Contact Info

Product

Resources

About