A Heuristic Feature Selection in Logistic Regression Modeling with Newton Raphson and Gradient Descent Algorithm

Handoyo, Samingun; Pradianti, Nandia; Nugroho, Waego Hadi; Akri, Yusnita Julyarni

doi:10.14569/ijacsa.2022.0130317

Cited by 4 publications

(6 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The results of different classification algorithms under different sample sets are not the same, and the selection of suitable classification algorithms to solve practical problems must first measure whether the selected classification algorithm is suitable, that is, how to evaluate the quality of the classification algorithm, generally depending on whether the classification algorithm results are reliable and stable and whether the performance of the classifier can be intuitively reflected, which is crucial in the evaluation of classification effect. The performance of the classifier is mainly measured by the following indicators [8]. The performance of the classifier is mainly measured by the following indicators.…”

Section: The Resultsmentioning

confidence: 99%

“…Its overly simplistic structure results in its lack of ability to extract keywords and understand their meaning and thereby infer keyword importance and calculate feature word distributions, which makes it prone to errors when dealing with weight adjustments. When dealing with the same type of corpus, the disadvantages of this method are very obvious, and it is easy to overwrite some keywords of the same type but with different meanings [5].…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

A spam classification method based on NB and SVM

Li¹

2023

Third International Conference on Machine Learning and Computer Application (ICMLCA 2022)

View full text Add to dashboard Cite

Section: The Resultsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

A spam classification method based on NB and SVM

Li¹

2023

Third International Conference on Machine Learning and Computer Application (ICMLCA 2022)

View full text Add to dashboard Cite

“…The selection of variables is only based on the level of dependence between two variables. The measuring scale of two variables evaluated for dependence lead to a kind of statistical test, namely the dependence between two categorical variables is evaluated by a chi-square test through a contingency table, and the dependence between numerical and categorical variables is evaluated by a one-way ANOVA test, and the dependence between 2 numerical variables is evaluated by correlation test [35]. The Pearson correlation formula given i.e.…”

Section: A Variable Selection and Data Formattingmentioning

confidence: 99%

“…The measuring scale of each variable will determine the appropriate evaluation method to check the independence between the two variables. The independence between the two numerical variables can be evaluated by their correlation value [35]. Table II presents the Spearmen correlation value between two predictor variables presented in the form of a matrix.…”

Section: A Evaluate Independency Among Predictorvariablesmentioning

confidence: 99%

Modeling Multioutput Response Uses Ridge Regression and MLP Neural Network with Tuning Hyperparameter through Cross Validation

Nugroho¹,

Handoyo²,

Hsieh³

et al. 2022

IJACSA

Self Cite

View full text Add to dashboard Cite

The multiple regression model is very popular among researchers in both field of social and science because it is easy to interpret and have a well-established theoretical framework. However, the multioutput multiple regression model is actually widely applied in the engineering field because in the industrial world there are many systems with multiple outputs. The ridge regression model and the Multi-Layer Perceptron (MLP) neural network model are representations of the predictive linear regression model and predictive non-linear regression model that are widely applied in the world of practice. This study aims to build multi-output models of a ridge regression model and an MLP neural network whose hyperparameters are determined by a grid search algorithm through the cross-validation method. The hyperparameter that produces the smallest RMSE value in the validation data is chosen as the hyperparameter to train both models on the training data. The hyperparameter in question is a combination of learning algorithms and alpha values (ridge regression), a combination of the number of hidden nodes and gamma values (MLP neural network). In the ridge regression model for alpha in the range between 0.1 and 0.7, the smallest RMSE is obtained for all learning algorithms used. While the MLP neural network model specifically obtained a combination of the number of nodes = 18 and gamma = 0.1 which produces the smallest RMSE. The ridge regression model with selected hyperparameters has better performance (in the RMSE and R2 value) than the MLP neural network model with selected hyperparameters, both on training and testing data.

show abstract

“…The training of the classification model aims to obtain a separating boundary that splits two class labels optimally. The comparison models including the type are the performance between the logistic regression with Newton Rapson and the gradient descent algorithm conducted by Handoyo et al [22], and the performance between the ridge logistic regression and the decision tree is done by Marji and Handoyo [23].…”

Section: Related Workmentioning

confidence: 99%

Finding the Best Performance of Bayesian and Naïve Bayes Models in Fraudulent Firms Classification through Varying Threshold

Widodo¹,

Handoyo²,

Irianto³

et al. 2022

IJETAE

View full text Add to dashboard Cite

—Fraud detection is the first step to preventing fraud committed by both individuals and organizations. The development of a high-performance classification model to detect fraud is an interesting topic in machine learning modeling. A finding of the best Bayesian and Naive Bayes classification models is a crucial issue because both models are simple and easily applied models in the fields of life and social sciences. This study aims to obtain the best performance of classification models developed based on probability concepts, namely Bayesian and Naive Bayes models. Adding a threshold value to the decision-making criteria of the two models is an effort expected to can find models that perform superiorly. Data on the auditing of fraudulent firms containing of 775 firms from various business sectors in Australia is used as a case study. The testing data consisting of 100 instances were taken by cluster random sampling with a proportion of 61 non-fraudulent and 39 fraudulent firms and the remaining instances as the training data. The best Bayesian model has an average accuracy of 84% obtained at a threshold value of 0.22. While the best Naive Bayes model has an average accuracy of 94% which is obtained in the 15 threshold values. Adding the threshold value has a significant impact on the performance of the Bayesian model, which can increase the average accuracy from 36% to 84%. On the other hand, the average accuracy of the Naive Bayes model only increased by 1%, from 93% to 94%. Performance measures Sensitivity, Specificity, F1 score, and ROC curve of the Naive Bayes model are also superior to the Bayesian model.

show abstract

A Heuristic Feature Selection in Logistic Regression Modeling with Newton Raphson and Gradient Descent Algorithm

Cited by 4 publications

References 0 publications

A spam classification method based on NB and SVM

A spam classification method based on NB and SVM

Modeling Multioutput Response Uses Ridge Regression and MLP Neural Network with Tuning Hyperparameter through Cross Validation

Finding the Best Performance of Bayesian and Naïve Bayes Models in Fraudulent Firms Classification through Varying Threshold

Contact Info

Product

Resources

About