Modeling Multioutput Response Uses Ridge Regression and MLP Neural Network with Tuning Hyperparameter through Cross Validation

Abstract

The multiple regression model is very popular among researchers in both field of social and science because it is easy to interpret and have a well-established theoretical framework. However, the multioutput multiple regression model is actually widely applied in the engineering field because in the industrial world there are many systems with multiple outputs. The ridge regression model and the Multi-Layer Perceptron (MLP) neural network model are representations of the predictive linear regression model and predictive non-linear regression model that are widely applied in the world of practice. This study aims to build multi-output models of a ridge regression model and an MLP neural network whose hyperparameters are determined by a grid search algorithm through the cross-validation method. The hyperparameter that produces the smallest RMSE value in the validation data is chosen as the hyperparameter to train both models on the training data. The hyperparameter in question is a combination of learning algorithms and alpha values (ridge regression), a combination of the number of hidden nodes and gamma values (MLP neural network). In the ridge regression model for alpha in the range between 0.1 and 0.7, the smallest RMSE is obtained for all learning algorithms used. While the MLP neural network model specifically obtained a combination of the number of nodes = 18 and gamma = 0.1 which produces the smallest RMSE. The ridge regression model with selected hyperparameters has better performance (in the RMSE and R2 value) than the MLP neural network model with selected hyperparameters, both on training and testing data.

Deep Learning Design Framework: High-Precision Front-End RFEH for Food Monitoring Applications

“…1) Multi-Output Regression (MOR): MOR is a machine learning technique designed to simultaneously predict multiple output variables, offering an advantage over traditional regression models by optimizing a joint objective function [52], [53], [54], [55], [56]. This approach is particularly effective when target variables have shared factors or interdependencies, thereby enhancing predictive accuracy in complex systems and scientific experiments.…”

“…It performs computations across layers, handling weighted sums and activation functions to produce non-linear outputs, which are essential for learning complex patterns. The MLP's ability to capture non-linear relationships, particularly through backpropagation, renders it highly effective in tasks like MOR, MLC, and MTS [52], [53], [57], [62]. Commonly, MLP utilizes the Adam optimization algorithm for its adaptive learning rate feature.…”

Finding the Best Performance of Bayesian and Naïve Bayes Models in Fraudulent Firms Classification through Varying Threshold

“…In a special case, the number of target features is greater than 1 feature, the model is known as the multi-output regression. Waego et al [20] developed the multi-output ridge regression to model the data in the health field. While multiple time series forecasting of agricultural data such as soybean prices is done by Handoyo and Chen [21] where the resulting fuzzy model is also categorized as a regression model in machine learning.…”