Variance-based sensitivity analysis: The quest for better estimators and designs between explorativity and economy

Piano, Samuele Lo; Ferretti, Federico; Puy, Arnald; Albrecht, Daniel; Saltelli, Andrea

doi:10.1016/j.ress.2020.107300

Cited by 33 publications

(23 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…. , X); although, the use of more than three matrices has been recently proven inefficient by Lo Piano et al [14].…”

Section: B (B (I)mentioning

confidence: 99%

“…• Form of the Test Function: Some of the most commonly used functions in a sensitivity analysis are the Ishigami and Homma [24], the Sobol' G and its variants [23,25], the Bratley and Fox [26] or the set of functions presented in Kucherenko et al [14,16,18,22,23]. Despite being analytically tractable, these functions capture only one possible interval of model behavior, and the effects of nonlinearities and nonadditivities is typically unknown in real models.…”

Section: Total-order Estimators and Uncertainties In The Benchmark Se...mentioning

confidence: 99%

“…This black box nature of models has become more of a concern in the last decades due to the increase in computational power and code complexity (which prevents the analyst from intuitively grasping the model's behavior [27]), and to the higher demand for model transparency [3,28,29]. This renders the functional form of the model similar to a random variable [21], something not accounted for by previous works [14,16,18,23].…”

Section: Total-order Estimators and Uncertainties In The Benchmark Se...mentioning

confidence: 99%

“…• Distribution of the Model Inputs: The large majority of benchmarking exercises assume uniformly-distributed inputs p(x) ∈ U (0, 1) k [14,16,23,33]. However, there is evidence that the accuracy of T i estimators might be sensitive to the underlying model input distributions, to the point of overturning the model input ranks [34,35].…”

Section: Total-order Estimators and Uncertainties In The Benchmark Se...mentioning

confidence: 99%

See 3 more Smart Citations

A Comprehensive Comparison of Total-Order Estimators for Global Sensitivity Analysis

Puy

Becker

Piano

et al. 2022

Int. J. UncertaintyQuantification

Self Cite

View full text Add to dashboard Cite

Sensitivity analysis helps identify which model inputs convey the most uncertainty to the model output. One of the most authoritative measures in global sensitivity analysis is the Sobol' total-order index, which can be computed with several different estimators. Although previous comparisons exist, it is hard to know which estimator performs best since the results are contingent on the benchmark setting defined by the analyst (the sampling method, the distribution of the model inputs, the number of model runs, the test function or model and its dimensionality, the weight of higher order effects, or the performance measure selected). Here we compare several total-order estimators in an eight-dimension hypercube, where these benchmark parameters are treated as random parameters. This arrangement significantly relaxes the dependency of the results on the benchmark design. We observe that the most accurate estimators are from Razavi and Gupta, Jansen, or Janon/Monod for factor prioritization, and from Jansen, Janon/Monod, or Azzini and Rosati for approaching the "true" total-order indices. The rest lag considerably behind. Our work helps analysts navigate myriad total-order formulae by reducing the uncertainty in the selection of the most appropriate estimator.

show abstract

“…. , X); although, the use of more than three matrices has been recently proven inefficient by Lo Piano et al [14].…”

Section: B (B (I)mentioning

confidence: 99%

Section: Total-order Estimators and Uncertainties In The Benchmark Se...mentioning

confidence: 99%

Section: Total-order Estimators and Uncertainties In The Benchmark Se...mentioning

confidence: 99%

Section: Total-order Estimators and Uncertainties In The Benchmark Se...mentioning

confidence: 99%

See 2 more Smart Citations

A Comprehensive Comparison of Total-Order Estimators for Global Sensitivity Analysis

Puy

Becker

Piano

et al. 2022

Int. J. UncertaintyQuantification

Self Cite

View full text Add to dashboard Cite

show abstract

“…It can be used with relatively small sample sizes and with datasets sampled with a generic Monte Carlo method, whereas the practical estimation of sensitivity indices with variance-based GSA typically requires numerical estimators (e.g., ; Online Resource 1, Sect. 1) and specific sampling designs to reduce computational cost (Lo Piano et al 2021). These sampling designs may be impossible to implement with commercial LCA software that only provide generic Monte Carlo sampling functionality.…”

Section: Introductionmentioning

confidence: 99%

Variance-based global sensitivity analysis and beyond in life cycle assessment: an application to geothermal heating networks

Jaxa-Rozen

Pratiwi

Trutnevyte

2021

Int J Life Cycle Assess

View full text Add to dashboard Cite

Purpose Global sensitivity analysis increasingly replaces manual sensitivity analysis in life cycle assessment (LCA). Variance-based global sensitivity analysis identifies influential uncertain model input parameters by estimating so-called Sobol indices that represent each parameter’s contribution to the variance in model output. However, this technique can potentially be unreliable when analyzing non-normal model outputs, and it does not inform analysts about specific values of the model input or output that may be decision-relevant. We demonstrate three emerging methods that build on variance-based global sensitivity analysis and that can provide new insights on uncertainty in typical LCA applications that present non-normal output distributions, trade-offs between environmental impacts, and interactions between model inputs. Methods To identify influential model inputs, trade-offs, and decision-relevant interactions, we implement techniques for distribution-based global sensitivity analysis (PAWN technique), spectral clustering, and scenario discovery (patient rule induction method: PRIM). We choose these techniques because they are applicable with generic Monte Carlo sampling and common LCA software. We compare these techniques with variance-based Sobol indices, using a previously published LCA case study of geothermal heating networks. We assess eight environmental impacts under uncertainty for three design alternatives, spanning different geothermal production temperatures and heating network configurations. Results In the application case on geothermal heating networks, PAWN distribution-based sensitivity indices generally identify influential model parameters consistently with Sobol indices. However, some discrepancies highlight the potentially misleading interpretation of Sobol indices on the non-normal distributions obtained in our analysis, where variance may not meaningfully describe uncertainty. Spectral clustering highlights groups of model results that present different trade-offs between environmental impacts. Compared to second-order Sobol interaction indices, PRIM then provides more precise information regarding the combinations of input values associated with these different groups of calculated impacts. PAWN indices, spectral clustering, and PRIM have a computational advantage because they yield stable results at relatively small sample sizes (n = 12,000), unlike Sobol indices (n = 100,000 for second-order indices). Conclusions We recommend adding these new techniques to global sensitivity analysis in LCA as they give more precise as well as additional insights on uncertainty regardless of the distribution of the model outputs. PAWN distribution-based global sensitivity analysis provides a computationally efficient assessment of input sensitivities as compared to variance-based global sensitivity analysis. The combination of clustering and scenario discovery enables analysts to precisely identify combinations of input parameters or uncertainties associated with different outcomes of environmental impacts.

show abstract

Efficient sensitivity analysis for biomechanical models with correlated inputs

Hilhorst,

Quicken,

van de Vosse

et al. 2023

Numer Methods Biomed Eng

View full text Add to dashboard Cite

In most variance‐based sensitivity analysis (SA) approaches applied to biomechanical models, statistical independence of the model input is assumed. However, often the model inputs are correlated. This might alter the interpretation of the SA results, which may severely impact the guidance provided during model development and personalization. Potential reasons for the infrequent usage of SA techniques that account for input correlation are the associated high computational costs, especially for models with many parameters, and the fact that the input correlation structure is often unknown. The aim of this study was to propose an efficient correlated global sensitivity analysis method by applying a surrogate model‐based approach. Furthermore, this article demonstrates how correlated SA should be interpreted and how the applied method can guide the modeler during model development and personalization, even when the correlation structure is not entirely known beforehand. The proposed methodology was applied to a typical example of a pulse wave propagation model and resulted in accurate SA results that could be obtained at a theoretically 27,000× lower computational cost compared to the correlated SA approach without employing a surrogate model. Furthermore, our results demonstrate that input correlations can significantly affect SA results, which emphasizes the need to thoroughly investigate the effect of input correlations during model development. We conclude that our proposed surrogate‐based SA approach allows modelers to efficiently perform correlated SA to complex biomechanical models and allows modelers to focus on input prioritization, input fixing and model reduction, or assessing the dependency structure between parameters.

show abstract

Variance-based sensitivity analysis: The quest for better estimators and designs between explorativity and economy

Cited by 33 publications

References 54 publications

A Comprehensive Comparison of Total-Order Estimators for Global Sensitivity Analysis

A Comprehensive Comparison of Total-Order Estimators for Global Sensitivity Analysis

Variance-based global sensitivity analysis and beyond in life cycle assessment: an application to geothermal heating networks

Efficient sensitivity analysis for biomechanical models with correlated inputs

Contact Info

Product

Resources

About