Selection of simplified models: II. Development of a model selection criterion based on mean squared error

Abstract

Simplified models (SMs) with a reduced set of parameters are used in many practical situations, especially when the available data for parameter estimation are limited. A variety of candidate models are often considered during the model formulation, simplification, and parameter estimation processes. We propose a new criterion to help modellers select the best SM, so that predictions with lowest expected mean squared error can be obtained. The effectiveness of the proposed criterion for selecting simplified nonlinear univariate and multivariate models is demonstrated using Monte-Carlo simulations and is compared with the effectiveness of the Bayesian Information Criterion (BIC).Des modèles simplifiés (MS) avec ensemble réduit de paramètres sont utilisés dans de nombreuses situations pratiques, particulièrement lorsque les données disponibles pour l'estimation des paramètres sont limitées. Divers modèles candidats sont souvent examinés durant la formulation du modèle, la simplification et l'estimation des paramètres. Nous proposons un nouveau critère pour aider les modélisateursà sélectionner le meilleur MS et obtenir des prédictions porteuses de l'erreur quadratique moyenne la plus faible. L'efficacité du critère proposé pour la sélection des modèles simplifiés non linéairesà variable uniques et multiples est démontrée en utilisant les simulations Monte-Carlo et comparéeà l'efficacité du critère bayesian des informations (BIC).

A Kinetic Model for Non‐Oxidative Thermal Degradation of Nylon 66

“…When estimability analysis was performed using all of the experimental settings (final study) 44 of the 56 parameters could be ranked before numerical problems were encountered. Wu's MSE‐based criterion21, 19 was used to determine that estimating the top 38 parameters should give the best model predictions for the preliminary study and the top 43 parameters should be estimated using all of the data. The parameters that were not selected for estimation are the initial [A2] for runs 4 and 6, initial [SB3] for all runs except run 3, initial [SB2] for all runs except run 3, and Δ H 6 .…”

“… Influence on the number of parameters estimated from the ranked list on the objective function value for preliminary study, (▴), and final parameter estimation study, (•). × indicates the number of parameters selected using Wu's method19, 21 (Wu, 2011). …”

“…It is not clear whether all of the model parameters can be estimated reliably from the available data. Recently, statistical techniques have been developed to aid modelers when estimating parameters in complex models using limited data 17–21. The objective of these estimability analysis and parameter selection techniques is to aid the modeler in determining which model parameters should be estimated from the available data, and which parameters should remain at their initial values.…”

“…In the current article, a mean‐squared‐error (MSE) criterion is used to determine the appropriate number of parameters to estimate from the ranked list 19, 21. Estimating too many parameters from the available data can lead to poor model predictions due to increased variance.…”

“…In this article, a dynamic kinetic model is developed to explain Schaffer's experimental data, which were obtained at relatively high temperatures (275–292 °C) and low water concentrations (10–80 mmol · kg −1 ) 4. Estimability analysis17, 18, 27 and a MSE‐based criterion19, 21 are used to determine which parameters can be estimated from the available data. Finally, the parameters and unknown initial conditions are estimated and model predictions are compared with experimental data.…”