Empirical survival Jensen-Shannon divergence as a goodness-of-Fit measure for maximum likelihood estimation and curve fitting

Abstract

The coefficient of determination, known as R 2 , is commonly used as a goodness-of-fit criterion for fitting linear models. R 2 is somewhat controversial when fitting nonlinear models, although it may be generalised on a case-by-case basis to deal with specific models such as the logistic model. Assume we are fitting a parametric distribution to a data set using, say, the maximum likelihood estimation method. A general approach to measure the goodness-of-fit of the fitted parameters, which is advocated herein, is to use a nonparametric measure for comparison between the empirical distribution, comprising the raw data, and the fitted model. In particular, for this purpose we put forward the Survival Jensen-Shannon divergence (SJS) and its empirical counterpart (ESJS) as a metric which is bounded, and is a natural generalisation of the Jensen-Shannon divergence. We demonstrate, via a straightforward procedure making use of the ESJS, that it can be used as part of maximum likelihood estimation or curve fitting as a measure of goodness-of-fit, including the construction of a confidence interval for the fitted parametric distribution. Furthermore, we show the validity of the proposed method with simulated data, and three empirical data sets.for nonlinear models, [5], where the author recommends to define R 2 as a comparison of a given model to the null model, claiming that this view allows for the generalisation of R 2 . Further, in [6] the inappropriateness of R 2 for nonlinear models is clearly demonstrated via a series of Monte Carlo simulations. In [7], a novel R 2 measure based on the Kullback-Leibler divergence [8] was proposed as a measure of goodness-of-fit for regression models in the exponential family. In addition, in [9] problems with using R 2 for assessing goodnessof-fit in linear mixed models with random effects were highlighted, and in [10] an improved extension was proposed in the context of both linear and generalised linear mixed models. Despite numerous proposals to address the issues with R 2 for nonlinear models, many of them are ad-hoc, and, as noted in [11], should be applied with caution. In summary, there seems to be a lack of general purpose goodness-of-fit measures that could be applied to nonlinear models, which is the main issue we attempt to redress with the ESJS.Alternative nonparametric methods have also been proposed. In particular, the Akaike information criterion (AIC) and its counterpart the Bayesian information criterion (BIC) [12,13], are widely used estimators for model selection. Both AIC and BIC are asymptotically valid maximum likelihood estimators, with penalty terms to discourage overfitting. We stress that goodness-of-fit measures how well a single model fits the observed data, while model selection compares the predictive accuracy of two models relative to each other [12,14]. The likelihood ratio test is also an established method for model selection between a null model and an alternative maximum likelihood model [15,16]. Despite the popularity of maximum likelihood meth...

A skew logistic distribution with application to modelling COVID-19 epidemic waves

“…We now turn to the evaluation of goodness-of-fit using the ESJS (empirical survival Jensen-Shannon divergence) [LK21,Lev21], which generalises the Jensen-Shannon divergence [Lin91] to survival functions, and the well-known KS2 (Kolmogorov-Smirnov two-sample test statistic) [GC21, Section 6.3]. In addition, we employ 95% bootstrap confidence intervals [ET93] In Table 3 we show the ESJS values for the four waves and the said improvements, while in Table 4 we show the corresponding KS2 values and improvements.…”

“…To fit of the skew logistic distribution to the time series data we employ maximum likelihood, and to evaluate the goodness-of-fit we make use of the empirical survival Jensen-Shannon divergence (ESJS) [LK21,Lev21] and the Kolmogorov-Smirnov two-sample test statistic (KS2) [GC21, Section 6.3]. We also supply 95% bootstrap confidence intervals [ET93] to assess the improvement of fitting the skew logistic distribution to the data over fitting the logistic or normal distributions.…”

A stochastic differential equation approach to the analysis of the 2017 and 2019 UK general election polls

“…[Dereich et al, 2012]), which is a general computational method for obtaining approximate numerical solutions to SDEs. We also make use of the Jensen-Shannon divergence (JSD) [Endres and Schindelin, 2003] as a goodness-of-fit measure [Levene and Kononovicius, 2019]. All computations were carried out using the Matlab software package.…”

A problem in human dynamics: modelling the population density of a social space

“…We can simulate the stochastic process corresponding to (3) using the Euler-Maruyama method [Sau13], which is a general computational method for obtaining approximate numerical solutions to SDEs. We will also make use of the Jensen-Shannon divergence (JSD) [ES03] as a goodness-of-fit measure [LK19]. All computations were carried out using the Matlab software package.…”