Convergence behavior of diffusion stochastic gradient descent algorithm

Barani, Fatemeh; Savadi, Abdorreza; Yazdi, Hadi Sadoghi

doi:10.1016/j.sigpro.2021.108014

Cited by 14 publications

(5 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This algorithm has the advantages of simplicity, low computational cost, fast convergence, and reliable effect [63,64]. Several scholars have already exploited the SGD model to address large-scale problems and obtain accurate findings [65,66]. However, the BN, DTable, and RBFN models also performed better, as the AUC value was greater than 0.8 on both the training and validation datasets.…”

Section: Discussionmentioning

confidence: 99%

GIS-Based Comparative Study of the Bayesian Network, Decision Table, Radial Basis Function Network and Stochastic Gradient Descent for the Spatial Prediction of Landslide Susceptibility

Huang

Ling

et al. 2022

Land

View full text Add to dashboard Cite

Landslides frequently occur along the eastern margin of the Tibetan Plateau, which poses a risk to the construction, maintenance, and transportation of the proposed Dujiangyan city to Siguniang Mountain (DS) railway, China. Therefore, four advanced machine learning models, namely, the Bayesian network (BN), decision table (DTable), radial basis function network (RBFN), and stochastic gradient descent (SGD), are proposed in this study to delineate landslide susceptibility zones. First, a landslide inventory map was randomly divided into 828 (75%) samples and 276 (25%) samples for training and validation, respectively. Second, the One-R technique was utilized to analyze the importance of 14 variables. Then, the prediction capability of the four models was validated and compared in terms of different statistical indices (accuracy (ACC) and Cohen’s kappa coefficient (k)) and the areas under the curve (AUC) in the receiver operating characteristic curve. The results showed that the SGD model performed best (AUC = 0.897, ACC = 80.98%, and k = 0.62), followed by the BN (AUC = 0.863, ACC = 78.80%, and k = 0.58), RBFN (AUC = 0.846, ACC = 77.36%, and k = 0.55), and DTable (AUC = 0.843, ACC = 76.45%, and k = 0.53) models. The susceptibility maps revealed that the DS railway segments from Puyang town to Dengsheng village are in high and very high-susceptibility zones.

show abstract

Section: Discussionmentioning

confidence: 99%

GIS-Based Comparative Study of the Bayesian Network, Decision Table, Radial Basis Function Network and Stochastic Gradient Descent for the Spatial Prediction of Landslide Susceptibility

Huang

Ling

et al. 2022

Land

View full text Add to dashboard Cite

show abstract

“…SGD is elucidated as an iterative optimization methodology, applicable to unconstrained optimization problems [20]. Its utility extends to identifying optimal parameter configurations for ML algorithms and optimizing objective functions with requisite smoothing properties [21]. Recognized for its simplicity and efficacy, SGD is particularly well-suited for fitting linear classifiers and regressors under convex loss functions [20].…”

Section: Sgdmentioning

confidence: 99%

Enhancing Anomaly-Based Intrusion Detection Systems: A Hybrid Approach Integrating Feature Selection and Bayesian Hyperparameter Optimization

Berbiche,

El Alami

2023

ISI

View full text Add to dashboard Cite

In the dynamically evolving landscape of cybersecurity, safeguarding IT infrastructures has emerged as an imperative to thwart the escalation of cyber-attacks. Anomaly-based Intrusion Detection Systems (IDS) play a pivotal role in identifying aberrant behaviours that elude conventional detection mechanisms. Nonetheless, these systems are not without their shortcomings, manifesting as elevated false alarm rates and a diminished efficacy in detecting sophisticated attacks. In response to these challenges, a hybrid approach, entailing Machine Learning (ML) techniques, was employed to augment the performance of anomaly-based IDS in terms of detection accuracy, False Positive (FP) Rate, and detection time. The approach encompassed a two-fold optimization strategy: initial feature selection predicated on feature importance derived from the XGBoost classifier, followed by Bayesian optimization (BO) for hyperparameter tuning. The optimization was conducted with respect to two objective functions, namely the ROC-AUC score and the Average Precision score, each serving to identify the optimal hyperparameters for their respective maximization. Classifiers, including Extreme Gradient Boosting (XGBoost), Random Forest (RF), and Stochastic Gradient Descent (SGD), were subjected to training under configurations encompassing both the hyperparameters resultant from BO and the default hyperparameters, the latter serving as reference models. Evaluation, conducted through a multifaceted metric analysis, substantiated the superiority of the optimized models over their reference counterparts, with the optimized XGBoost models demonstrating the most commendable performance. This paradigm offers a promising avenue for enhancing detection precision and mitigating false alarms, thereby fortifying the security of computer systems.

show abstract

“…For deterministic optimization algorithms, such as sequential minimal optimization (SMO) [8] and stochastic gradient descent (SGD) [9], if the objective function is discontinuous and nondifferentiable, their convergence speed is usually slow and they will easily fall into the local optimum. As a stochastic optimization method, the swarm intelligence optimization methods introduce a brand new path to solve global optimization problems by taking advantage of randomness.…”

Section: Introductionmentioning

confidence: 99%

FoSSA Optimization-Based SVM Classifier for the Recognition of Partial Discharge Patterns in HV Cables

Sun

Meng

Dong

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

In order to enhance the classification accuracy and the generalization performance of the SVM classifier in cable partial discharge (PD) pattern recognition, a firefly optimized sparrow search algorithm (FoSSA) is proposed to optimize its kernel function parameters and penalty factors. First, the Circle-Gauss hybrid mapping model is employed in the population initialization stage of the sparrow search algorithm (SSA) to eliminate the uneven population distribution of random mapping. Sparrows tend to fall into local extremums during the search process, while the firefly algorithm has a fast optimization speed and strong local search ability. Thus, a firefly disturbance is added in the sparrow search process, and the fitness value is recalculated to update the sparrow position to enhance the sparrow’s local optimization ability and accuracy. Finally, based on the SSA, a dynamic step-size strategy is adopted to make the step size dynamically decrease with the number of iterations and improve the accuracy of convergence. Six benchmark functions are employed to evaluate the optimization performance of the FoSSA quantitatively. Experiment results show that the recognition accuracy of the PD patterns using the SVM optimized by the FoSSA could reach 97.5%.

show abstract

Convergence behavior of diffusion stochastic gradient descent algorithm

Cited by 14 publications

References 19 publications

GIS-Based Comparative Study of the Bayesian Network, Decision Table, Radial Basis Function Network and Stochastic Gradient Descent for the Spatial Prediction of Landslide Susceptibility

GIS-Based Comparative Study of the Bayesian Network, Decision Table, Radial Basis Function Network and Stochastic Gradient Descent for the Spatial Prediction of Landslide Susceptibility

Enhancing Anomaly-Based Intrusion Detection Systems: A Hybrid Approach Integrating Feature Selection and Bayesian Hyperparameter Optimization

FoSSA Optimization-Based SVM Classifier for the Recognition of Partial Discharge Patterns in HV Cables

Contact Info

Product

Resources

About