Improved high-dimensional regression models with matrix approximations applied to the comparative case studies with support vector machines

Roozbeh, Mahdi; Babaie-Kafaki, Saman; Aminifard, Zohre

doi:10.1080/10556788.2021.2022144

Cited by 8 publications

(4 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As is known, obtaining the optimal value of the ridge parameter is not generally simple and it depends on the criterion used in the prediction problem and dataset. Furthermore, the ridge regression method combats the multicollinearity problem and estimates the parameters by adding shrinkage parameter k to the diagonal elements of X X, which leads to distortion of the data [28,29]. LASSO is based on balancing the opposing factors of bias and variance to build the most predictive model.…”

Section: Discussionmentioning

confidence: 99%

Generalized Support Vector Regression and Symmetry Functional Regression Approaches to Model the High-Dimensional Data

Roozbeh,

Rouhi,

Mohamed

et al. 2023

Symmetry

Self Cite

View full text Add to dashboard Cite

The analysis of the high-dimensional dataset when the number of explanatory variables is greater than the observations using classical regression approaches is not applicable and the results may be misleading. In this research, we proposed to analyze such data by introducing modern and up-to-date techniques such as support vector regression, symmetry functional regression, ridge, and lasso regression methods. In this study, we developed the support vector regression approach called generalized support vector regression to provide more efficient shrinkage estimation and variable selection in high-dimensional datasets. The generalized support vector regression can improve the performance of the support vector regression by employing an accurate algorithm for obtaining the optimum value of the penalty parameter using a cross-validation score, which is an asymptotically unbiased feasible estimator of the risk function. In this regard, using the proposed methods to analyze two real high-dimensional datasets (yeast gene data and riboflavin data) and a simulated dataset, the most efficient model is determined based on three criteria (correlation squared, mean squared error, and mean absolute error percentage deviation) according to the type of datasets. On the basis of the above criteria, the efficiency of the proposed estimators is evaluated.

show abstract

Section: Discussionmentioning

confidence: 99%

Generalized Support Vector Regression and Symmetry Functional Regression Approaches to Model the High-Dimensional Data

Roozbeh,

Rouhi,

Mohamed

et al. 2023

Symmetry

Self Cite

View full text Add to dashboard Cite

show abstract

“…where is the penalty parameter and stands for the spectral condition number [26,27]. for an arbitrary positive definite matrix .…”

Section: An Improved Model For the Nonnegative Matrix Factorization P...mentioning

confidence: 99%

Untitled

2023

AnnalsARSciInfo

View full text Add to dashboard Cite

Despite computational superiorities, some traditional conjugate gradient algorithms such as Polak-Ribiére-Polyak and Hestenes-Stiefel methods generally fail to guarantee the descent condition. Here, in a matrix viewpoint, spectral versions of such methods are developed which fulfill the descent condition. The convergence of the given spectral algorithms is argued briefly. Afterwards, we propose an improved version of the nonnegative matrix factorization problem by adding penalty terms to the model, for controlling the condition number of one of the factorization elements. Finally, the computational merits of the method are examined using a set of CUTEr test problems as well as some random nonnegative matrix factorization models. The results typically agree with our analytical spectrum.

show abstract

“…When the amount of hidden layers is maximal than the amount of training samples, β is denoted as follows [20]:…”

Section: Elm Based Classificationmentioning

confidence: 99%

Chicken Swarm-Based Feature Subset Selection with Optimal Machine Learning Enabled Data Mining Approach

et al. 2022

View full text Add to dashboard Cite

Data mining (DM) involves the process of identifying patterns, correlation, and anomalies existing in massive datasets. The applicability of DM includes several areas such as education, healthcare, business, and finance. Educational Data Mining (EDM) is an interdisciplinary domain which focuses on the applicability of DM, machine learning (ML), and statistical approaches for pattern recognition in massive quantities of educational data. This type of data suffers from the curse of dimensionality problems. Thus, feature selection (FS) approaches become essential. This study designs a Feature Subset Selection with an optimal machine learning model for Educational Data Mining (FSSML-EDM). The proposed method involves three major processes. At the initial stage, the presented FSSML-EDM model uses the Chicken Swarm Optimization-based Feature Selection (CSO-FS) technique for electing feature subsets. Next, an extreme learning machine (ELM) classifier is employed for the classification of educational data. Finally, the Artificial Hummingbird (AHB) algorithm is utilized for adjusting the parameters involved in the ELM model. The performance study revealed that FSSML-EDM model achieves better results compared with other models under several dimensions.

show abstract

Improved high-dimensional regression models with matrix approximations applied to the comparative case studies with support vector machines

Cited by 8 publications

References 21 publications

Generalized Support Vector Regression and Symmetry Functional Regression Approaches to Model the High-Dimensional Data

Generalized Support Vector Regression and Symmetry Functional Regression Approaches to Model the High-Dimensional Data

Untitled

Chicken Swarm-Based Feature Subset Selection with Optimal Machine Learning Enabled Data Mining Approach

Contact Info

Product

Resources

About