Meta-Optimization of Bias-Variance Trade-Off in Stochastic Model Learning

Aotani, Takumi; Kobayashi, Taisuke; Sugimoto, Kenji

doi:10.1109/access.2021.3125000

Cited by 12 publications

(4 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A high-variance problem occurs when a machine learning model is trained on a dataset with unvalued features or noise; the model trains very well but provides incorrect results when applied to new data. When the variance in the machine learning model is high and the bias is low, the machine learning model suffers from the underfitting problem [21][22]. Figure 3.0 underlines the high bias that is described in the following.…”

Section: Bias-variance Tradeoffmentioning

confidence: 89%

An Optimal Solution to the Overfitting and Underfitting Problem of Healthcare Machine Learning Models

Anil,

Singh

2023

J Syst Eng Inf Technol

View full text Add to dashboard Cite

In the current technological era, artificial intelligence is becoming increasingly popular. Machine learning, as the branch of AI is taking charge in every field such as healthcare, the Stock market, Automation, Robotics, Image Processing, and so on. In the current scenario, machine learning and/or deep learning are becoming very popular in medical science for disease prediction. Much research is underway in the form of disease prediction models by machine learning. To ensure the performance and accuracy of the machine learning model, it is important to keep some basic things in mind during training. The machine learning model has several issues which must be rectified duration of the training of the model so that the learning model works efficiently such as model selection, parameter tuning, dataset splitting, cross-validation, bias-variance tradeoff, overfitting, underfitting, and so on. Under- and over-fitting are the two main issues that affect machine learning models. This research paper mainly focuses on minimizing and/or preventing the problem of overfitting and underfitting machine learning models.

show abstract

Section: Bias-variance Tradeoffmentioning

confidence: 89%

An Optimal Solution to the Overfitting and Underfitting Problem of Healthcare Machine Learning Models

Anil,

Singh

2023

J Syst Eng Inf Technol

View full text Add to dashboard Cite

show abstract

“…In fact, in highway-env simulations, we observed cases where other blue cars could not be reconstructed as in the standard VAE, depending on the value of q 2 . A framework that is adaptive or robust to this trade-off, such as meta-optimization [32] or ensemble learning with multiple combinations of hyperparameters [33], would be useful.…”

Section: Discussion For Future Workmentioning

confidence: 99%

Sparse Representation Learning with Modified q-VAE towards Minimal Realization of World Model

Kobayashi¹,

Watanuki²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Extraction of low-dimensional latent space from high-dimensional observation data is essential to construct a real-time robot controller with a world model on the extracted latent space. However, there is no established method for tuning the dimension size of the latent space automatically, suffering from finding the necessary and sufficient dimension size, i.e. the minimal realization of the world model. In this study, we analyze and improve Tsallis-based variational autoencoder (q-VAE), and reveal that, under an appropriate configuration, it always facilitates making the latent space sparse. Even if the dimension size of the pre-specified latent space is redundant compared to the minimal realization, this sparsification collapses unnecessary dimensions, allowing for easy removal of them. We experimentally verified the benefits of the sparsification by the proposed method that it can easily find the necessary and sufficient six dimensions for a reaching task with a mobile manipulator that requires a six-dimensional state space. Moreover, by planning with such a minimal-realization world model learned in the extracted dimensions, the proposed method was able to exert a more optimal action sequence in real-time, reducing the reaching accomplishment time by around 20 %.

show abstract

“…Therefore, in order to avoid overfitting, we used complexity regularization to address the bias–variance trade-off. The interested reader may be referred to references [ 41 , 42 , 43 ] for a detailed explanation on the bias–variance trade-off and potential model selection options (e.g., hold-out, k -fold cross-validation, structural risk minimization, complexity regularization, and information criteria).…”

Section: Proposed Approach Materials and Methodsmentioning

confidence: 99%

Applied Machine Learning in Industry 4.0: Case-Study Research in Predictive Models for Black Carbon Emissions

Rubio‐Loyola

Paul-Fils

2022

Sensors

View full text Add to dashboard Cite

Industry 4.0 constitutes a major application domain for sensor data analytics. Industrial furnaces (IFs) are complex machines made with special thermodynamic materials and technologies used in industrial production applications that require special heat treatment cycles. One of the most critical issues while operating IFs is the emission of black carbon (EoBC), which is due to a large number of factors such as the quality and amount of fuel, furnace efficiency, technology used for the process, operation practices, type of loads and other aspects related to the process conditions or mechanical properties of fluids at furnace operation. This paper presents a methodological approach to predict EoBC during the operation of IFs with the use of predictive models of machine learning (ML). We make use of a real data set with historical operation to train ML models, and through evaluation with real data we identify the most suitable approach that best fits the characteristics of the data set and implementation constraints in real production environments. The evaluation results confirm that it is possible to predict the undesirable EoBC well in advance, by means of a predictive model. To the best of our knowledge, this paper is the first approach to detail machine-learning concepts for predicting EoBC in the IF industry.

show abstract

Meta-Optimization of Bias-Variance Trade-Off in Stochastic Model Learning

Cited by 12 publications

References 21 publications

An Optimal Solution to the Overfitting and Underfitting Problem of Healthcare Machine Learning Models

An Optimal Solution to the Overfitting and Underfitting Problem of Healthcare Machine Learning Models

Sparse Representation Learning with Modified q-VAE towards Minimal Realization of World Model

Applied Machine Learning in Industry 4.0: Case-Study Research in Predictive Models for Black Carbon Emissions

Contact Info

Product

Resources

About