A New Loss Function for CNN Classifier Based on Predefined Evenly-Distributed Class Centroids

Zhu, Qiuyu; Zhang, Pengju; Wang, Zhengyong; Ye, Xin

doi:10.1109/access.2019.2960065

Cited by 37 publications

(23 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Where, N, in this instance, is the sums of iteration,f_i is the training loss values and y_i is the testing loss values. Consequently, MSE is calculated, as presented in (2) [36], [37].…”

Section: Mean Squared Errormentioning

confidence: 99%

Comparative analysis of augmented datasets performances of age invariant face recognition models

et al. 2021

View full text Add to dashboard Cite

The popularity of face recognition systems has increased due to their non-invasive method of image acquisition, thus boasting the widespread applications. Face ageing is one major factor that influences the performance of face recognition algorithms. In this study, the authors present a comparative study of the two most accepted and experimented face ageing datasets (FG-Net and morph II). These datasets were used to simulate age invariant face recognition (AIFR) models. Four types of noises were added to the two face ageing datasets at the preprocessing stage. The addition of noise at the preprocessing stage served as a data augmentation technique that increased the number of sample images available for deep convolutional neural network (DCNN) experimentation, improved the proposed AIFR model and the trait aging features extraction process. The proposed AIFR models are developed with the pre-trained Inception-ResNet-v2 deep convolutional neural network architecture. On testing and comparing the models, the results revealed that FG-Net is more efficient over Morph with an accuracy of 0.15%, loss function of 71%, mean square error (MSE) of 39% and mean absolute error (MAE) of -0.63%.

show abstract

“…Where, N, in this instance, is the sums of iteration,f_i is the training loss values and y_i is the testing loss values. Consequently, MSE is calculated, as presented in (2) [36], [37].…”

Section: Mean Squared Errormentioning

confidence: 99%

Comparative analysis of augmented datasets performances of age invariant face recognition models

et al. 2021

View full text Add to dashboard Cite

show abstract

“…All experimental results show that our method has better classification accuracy than classic Softmax loss, AM-Softmax, and PEDCC-loss. Here we set two sets hyperparameter s = 7.5, m = 0.35, and s = 10, m = 0.5, which is following the original PEDCC-loss [9]. The hyperparameter s is used to let the network be trainable.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…Although the Softmax+cross-entropy loss function has achieved great success in many computer vision fields as a norm of neural network, some improvements have been made by researchers to let the features learned by the convolutional neural network more distinctive between classes. Loss functions such as center-loss [5], L-Softmax [6], A-Softmax [7], AM-Softmax (Additive Margin) [8], PEDCC-loss (Pre-defined Evenly-Distributed Class Centroids) [9], etc. can improve the final accuracy of models when trained for face classification and validation.…”

Section: Introductionmentioning

confidence: 99%

Improving Classification Performance of Softmax Loss Function Based on Scalable Batch-Normalization

Zhu

Zhang

et al. 2020

Applied Sciences

Self Cite

View full text Add to dashboard Cite

Convolutional neural networks (CNNs) have made great achievements on computer vision tasks, especially the image classification. With the improvement of network structure and loss functions, the performance of image classification is getting higher and higher. The classic Softmax + cross-entropy loss has been the norm for training neural networks for years, which is calculated from the output probability of the ground-truth class. Then the network’s weight is updated by gradient calculation of the loss. However, after several epochs of training, the back-propagation errors usually become almost negligible. For the above considerations, we proposed that batch normalization with adjustable scale could be added after network output to alleviate the problem of vanishing gradient problem in deep learning. The experimental results show that our method can significantly improve the final classification accuracy on different network structures, and is also better than many other improved classification Loss.

show abstract

“…Due to the solidifying characteristics, PEDCC provides a unique research perspective for the solution of key problems in the field of pattern recognition, such as interpretable supervised/unsupervised learning, incremental learning, uncertainty analysis an so on. Now, PEDCC has been used in CNN classifiers [12], classification autoencoders [13], clustering [14], semi-supervised learning [15], etc. Although PEDCC has shown some excellent characteristics and has been well applied in some aspects, the mathematical generation method and related characteristics of PEDCC have not been well studied, which hinders its further application.…”

Section: Introductionmentioning

confidence: 99%

Generation and Frame Characteristics of Predefined Evenly-Distributed Class Centroids for Pattern Classification

et al. 2021

Self Cite

View full text Add to dashboard Cite

Predefined evenly-distributed class centroids (PEDCC) can be widely used in models and algorithms of pattern classification, such as CNN classifiers, classification autoencoders, clustering, and semi-supervised learning, etc. Its basic idea is to predefine the class centers, which are evenly-distributed on the unit hypersphere in feature space, to maximize the inter-class distance. The previous method of generating PEDCC uses an iterative algorithm based on a charge model. The generated class centers will have some errors with the theoretically evenly-distributed points, and the generation time is long. This paper takes advantage of regular polyhedron in high-dimensional space and the evenly distributed points on the n dimensional hypersphere to generate PEDCC mathematically. Then, we discussed the basic and extended characteristics of the frames formed by PEDCC, and some meaningful conclusions are obtained. Finally, the effectiveness of the new algorithm and related conclusions are proved by experiments. The mathematical analysis and experimental results of this paper can provide a theoretical tool for using PEDCC to solve the key problems in the field of pattern recognition, such as interpretable supervised/unsupervised learning, incremental learning, uncertainty analysis and so on.INDEX TERMS Predefined evenly-distributed class centroids, polyhedron in high-dimensional space, pattern classification, CNN classifiers, frame theory.

show abstract

A New Loss Function for CNN Classifier Based on Predefined Evenly-Distributed Class Centroids

Cited by 37 publications

References 17 publications

Comparative analysis of augmented datasets performances of age invariant face recognition models

Comparative analysis of augmented datasets performances of age invariant face recognition models

Improving Classification Performance of Softmax Loss Function Based on Scalable Batch-Normalization

Generation and Frame Characteristics of Predefined Evenly-Distributed Class Centroids for Pattern Classification

Contact Info

Product

Resources

About