GT-SGD: A Novel Gradient Synchronization Algorithm in Training Distributed Recurrent Neural Network Language Models

Zhang, Xiaoci; Gu, Naijie; Yasrab, Robail; Ye, Hong

doi:10.1109/nana.2017.22

Cited by 1 publication

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SGD is an iterative optimization algorithm that is often used to solve and optimize model parameters of machine learning algorithms. SGD is a deformed form of the gradient descent algorithm, which has been successfully applied to text classification [33] and large-scale sparse machine learning problems in natural language processing [34]- [35]. The gradient is to obtain the partial derivative of the unknown parameters of a multivariate function and obtain the vector composed of these partial derivative functions.…”

Section: ) Fault Classification and Prediction Model Based On Stochastic Gradient Descentmentioning

confidence: 99%

Power System Fault Classification and Prediction Based on a Three-Layer Data Mining Structure

Wang

et al. 2020

IEEE Access

View full text Add to dashboard Buy / Rent full text

In traditional fault diagnosis methods in power systems, it is difficult to accurately classify and predict the types of faults. With the emergence of big data technology, the fault classification and prediction methods based on big data analysis and processing have been applied in power systems. To make the classification and prediction of the fault types more accurate, this paper proposes a hybrid data mining method for power system fault classification and prediction based on clustering, association rules and stochastic gradient descent. This method uses a three-layer data mining model: The first layer uses the K-means clustering algorithm to preprocess the original fault data source, and it proposes to use self-encoding to simplify the data form. The second layer effectively eliminates the data that have little impact on the prediction results by using association rules, and the highly correlated data are mined to become the regression training data. The third layer first uses the cross-validation method to obtain the optimal parameters of each fault model, and then, it uses stochastic gradient descent for data regression training to obtain a classification and prediction model for each fault type. Finally, a verification example shows that compared with a single data mining algorithm model, the proposed method is more comparative in terms of the data mining, and the established power system fault classification and prediction model has global optimality and higher prediction accuracy, which has a certain feasibility for real-time online power system fault classification and prediction. This method reduces the disturbances from low-impact or irrelevant data by mining the fault data three times, and it uses cross-validation to optimize the multiple regression parameters of the regression model to solve the problems of low accuracy, large errors and easily falling into a local optimum, given the conduct of fault classification and prediction.

show abstract

Section: ) Fault Classification and Prediction Model Based On Stochastic Gradient Descentmentioning

confidence: 99%

Power System Fault Classification and Prediction Based on a Three-Layer Data Mining Structure

Wang

et al. 2020

IEEE Access

View full text Add to dashboard Buy / Rent full text

show abstract

GT-SGD: A Novel Gradient Synchronization Algorithm in Training Distributed Recurrent Neural Network Language Models

Cited by 1 publication

References 11 publications

Power System Fault Classification and Prediction Based on a Three-Layer Data Mining Structure

Power System Fault Classification and Prediction Based on a Three-Layer Data Mining Structure

Contact Info

Product

Resources

About