Comparison of two sets of Monte Carlo estimators of Sobol’ indices

Abstract

This study compares the performances of two sampling-based strategies for the simultaneous estimation of the first-and total-order variance-based sensitivity indices (a.k.a. Sobol' indices). The first strategy corresponds to the current approach employed by practitioners and recommended in the literature. The second one was only recently introduced by the first and last authors of the present article. Both strategies rely on different estimators of first-and total-order Sobol' indices. The asymptotic normal variances of the two sets of estimators are established and their accuracies are compared theoretically and numerically. The results show that the new strategy outperforms the current one. The global sensitivity analysis of the radiative forcing model of sulfur aerosols is performed with the new strategy. The results confirm that in this model interactions are important and only one input variable is irrelevant.

How Meaningful is the Elite Quality Index Ranking?

“…In order to identify the input factors that exert the largest influence on the ranking, we conducted a variance-based SA. We estimated Sobol' first-and total effect sensitivity indices using the state of the art Azzini et al (2020) estimator. The SA revealed that a potential measurement error, the omission of individual indicators, different conceptual optima used for selected EQx indicators, as well as the aggregation method at the sub-index level have negligible effects on the country ranking.…”

“…They empirically compare eight MC-based estimators for the total-sensitivity index. The estimator developed by Azzini et al (2020) is one of the best performing, both when the goal is to rank input factors according to their contribution to model output variance (factor prioritisation setting), as well as when the aim is to approximate the "true" indices (factor fixing setting) (p.12). Thus, the Azzini estimator is recommended by Puy et al (2021b, p. 19), since it is both efficient and robust to a wide range of practical situations.…”

“…Performing a variance-based SA, the first-order sensitivity index S i as well as the total effect sensitivity index S Ti will be computed and analysed. We estimate S i and S Ti based on the formulas presented by Azzini et al (2020) 3…”

“…The search for estimators of Sobol' indices, that are both efficient and robust, is currently an active field of research. One recommended estimator according to Puy et al (2021b, p. 19), is the Azzini estimator, recently presented by Azzini et al (2020), as it is efficient and robust to a wide range of practical situations. To the best of our knowledge, this paper provides the first application of the Azzini estimator of Sobol' indices to assess the performance of an index.…”

Models with higher effective dimensions tend to produce more uncertain estimates

“…The model f runs row-wise in each matrix, and we use the model output y to compute the coefficient of variance and the sensitivity indices. We compute S i and T i with the Azzini estimators ( 66 )Si=2truetrue∑v=1Nfalse(ffalse(BAfalse(ifalse)false)v−ffalse(bold-italicBfalse)vfalse)false(ffalse(bold-italicAfalse)v−ffalse(ABfalse(ifalse)false)vfalse)truetrue∑v=1Nfalse[false(ffalse(bold-italicAfalse)v−ffalse(bold-italicBfalse)vfalse)2+false(ffalse(BAfalse(ifalse)false)v−ffalse(ABfalse(ifalse)false)vfalse)2false]Ti=truetrue∑v=1Nfalse[ffalse(bold-italicBfalse)v−ffalse(BAfalse(ifalse)f...…”