Selection of simplified models: I. Analysis of model‐selection criteria using mean‐squared error

Abstract

Mean-squared error (MSE) is used to analyse nine commonly used model-selection criteria (MSC) for their performance when selecting simplified models (SMs). Expressions are derived to enable exact calculations of the probability that a particular MSC will select a SM. For several common MSC, the relative propensities to select SMs are independent of model structure and data. It is shown that MSC that are effective in preventing overfitting are prone to underfitting when information content of the data is low. In a subsequent article, results are extended to develop a new MSE-based MSC for selecting nonlinear multi-response SMs.L'erreur quadratique moyenne (EQM) est utilisée pour analyser neuf critères de sélection de modèles (CSM) couramment utilisés pour leur rendement lors de la sélection de modèles simplifiés (MS). Des expressions sont trouvées pour permettre des calculs exacts de la probabilité qu'un CSM particulier choisisse un MS. Pour plusieurs CSM communs, les propensions relatives pour choisir des MS sont indépendantes de la structure et des données des modèles. On a démontré que les CSM qui sont efficaces pour empêcher le surajustement sont enclins au sous-ajustement lorsque le contenu de l'information des données est faible. Dans un article ultérieur, les résultats sontétendus pour créer un nouveau CSM fondé sur l'EQM pour la sélection de MS non linéairesà réponses multiples.

Application of Parameter Selection and Estimation Techniques in a Thermal Styrene Polymerization Model

“…The chemical mechanisms considered in this work are shown in Table 1 and initial literature values of the associated kinetic rate coefficients are shown in Table 2. A second objective for this work is to demonstrate how recently‐developed techniques can be used to determine which model parameters should remain at their literature values and which should be adjusted to improve the model fit to the data 9–11…”

“…Estimating too many parameters can lead to large variance in model predictions and estimating too few parameters can lead to large bias because important parameters are fixed at incorrect values. Recently, Wu et al developed a statistical method that can be used to determine the optimal number of parameters to estimate from the ranked list 66. Wu's method minimizes the expected mean‐squared error in the model predictions by considering the tradeoff between variance and bias.…”

A Kinetic Model for Non‐Oxidative Thermal Degradation of Nylon 66

“…Results of the parameter ranking and selection methods that were used depend on the initial parameters values and scaling factors employed 20, 21. To test the robustness of the ranking results in Table 10, we investigated the effects of changing the initial guesses and scaling factors.…”

“…It is not clear whether all of the model parameters can be estimated reliably from the available data. Recently, statistical techniques have been developed to aid modelers when estimating parameters in complex models using limited data 17–21. The objective of these estimability analysis and parameter selection techniques is to aid the modeler in determining which model parameters should be estimated from the available data, and which parameters should remain at their initial values.…”

“…Estimating too many parameters from the available data can lead to poor model predictions due to increased variance. Estimating too few parameters, and leaving the remaining parameters at their incorrect initial guesses, can lead to poor model predictions due to increased bias 20. One objective of the current modeling work is to determine whether all of the parameters can be estimated from Schaffer's4 data and, if not, to obtain the best possible estimates of the parameters that should be estimated.…”