Use of an excess variance approach for the certification of reference materials by interlaboratory comparison

Rivier, Cédric; Désenfant, Michèle; Crozet, Marielle; Rigaux, Corinne; Roudil, D.; Tufféry, B.; Ruas, Alexandre

doi:10.1007/s00769-014-1066-3

Cited by 10 publications

(7 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If χ 2 w is too small, the u(e i ) are over-estimated and should be reconsidered, or the benchmarked method is over-fitting the data, which is unlikely, unless the method is parametric and has been calibrated on this same dataset. If χ 2 w is too large, there is an excess of variance in the E M error set [44][45][46] . In the typical benchmarking of computational chemistry methods, this is generally the case because of the extraneous dispersion due to model errors.…”

Section: Appendices Appendix A: Estimation Of the Mean Value And Its mentioning

confidence: 99%

“…converges properly to the standard limit when the reference data errors become negligible before the model errors. The model error variance σ 2 can be estimated by decomposing the total variance of the errors into the variance of model errors plus the mean variance of the data (known as Cochran's ANOVA estimate 44,46 ) var(e) = σ 2 + 1 N N j=1 u(e j ) 2 (A10)…”

Section: Appendices Appendix A: Estimation Of the Mean Value And Its mentioning

confidence: 99%

“…This variance analysis ensures that χ 2 w is correct. Note that other reweighting schemes exist 44,46 , but Cochran's is the simplest. Besides, reweighting methods are iterative: σ depends on e, which itself depends on σ.…”

Section: Appendices Appendix A: Estimation Of the Mean Value And Its mentioning

confidence: 99%

“…the errors into the variance of model errors plus the mean variance of the data (known as Cochran's ANOVA estimate 44,46 )…”

Section: Appendices Appendix A: Estimation Of the Mean Value And Its ...mentioning

confidence: 99%

See 3 more Smart Citations

Probabilistic performance estimators for computational chemistry methods: Systematic improvement probability and ranking probability matrix. I. Theory

Pernot

Savin

2020

The Journal of Chemical Physics

View full text Add to dashboard Cite

The comparison of benchmark error sets is an essential tool for the evaluation of theories in computational chemistry. The standard ranking of methods by their Mean Unsigned Error is unsatisfactory for several reasons linked to the non-normality of the error distributions and the presence of underlying trends. Complementary statistics have recently been proposed to palliate such deficiencies, such as quantiles of the absolute errors distribution or the mean prediction uncertainty. We introduce here a new score, the systematic improvement probability (SIP), based on the direct system-wise comparison of absolute errors. Independently of the chosen scoring rule, the uncertainty of the statistics due to the incompleteness of the benchmark data sets is also generally overlooked. However, this uncertainty is essential to appreciate the robustness of rankings. In the present article, we develop two indicators based on robust statistics to address this problem: P inv , the inversion probability between two values of a statistic, and P r , the ranking probability matrix. We demonstrate also the essential contribution of the correlations between error sets in these scores comparisons.

show abstract

Section: Appendices Appendix A: Estimation Of the Mean Value And Its mentioning

confidence: 99%

Section: Appendices Appendix A: Estimation Of the Mean Value And Its mentioning

confidence: 99%

Section: Appendices Appendix A: Estimation Of the Mean Value And Its mentioning

confidence: 99%

“…the errors into the variance of model errors plus the mean variance of the data (known as Cochran's ANOVA estimate 44,46 )…”

Section: Appendices Appendix A: Estimation Of the Mean Value And Its ...mentioning

confidence: 99%

See 2 more Smart Citations

Probabilistic performance estimators for computational chemistry methods: Systematic improvement probability and ranking probability matrix. I. Theory

Pernot

Savin

2020

The Journal of Chemical Physics

View full text Add to dashboard Cite

show abstract

“…From[5].metrology committee CCQM (Consultative committee for amount of substance). Uncertainties are determined accordingly[9].…”

mentioning

confidence: 99%

Metrology applications to D&D issues: issues at stake for INSIDER European project

Roudil

Crozet

Picart

et al. 2020

EPJ Nuclear Sci. Technol.

View full text Add to dashboard Cite

Nuclear metrology is an essential aspect to consider for further improvements of the initial characterization of sites under decommissioning. The H2020 Euratom project INSIDER in June 2017 aims at improving the management of contaminated materials arising from decommissioning and dismantling (D&D) operations by proposing an integrated methodology for radiological characterization. This methodology is based on advanced statistical processing and modelling, coupled with adapted or innovative measurement methods. A metrological approach supports the qualification of this integrated methodology with a concrete application to real projects representative of the use cases identified in the project. Assessment of the outcomes will be used for providing recommendations and guidance resulting in pre-standardization texts.

show abstract

Bayesian analysis of homogeneity studies in the production of reference materials

Veen¹

2017

Accred Qual Assur

View full text Add to dashboard Cite

Use of an excess variance approach for the certification of reference materials by interlaboratory comparison

Cited by 10 publications

References 26 publications

Probabilistic performance estimators for computational chemistry methods: Systematic improvement probability and ranking probability matrix. I. Theory

Probabilistic performance estimators for computational chemistry methods: Systematic improvement probability and ranking probability matrix. I. Theory

Metrology applications to D&D issues: issues at stake for INSIDER European project

Bayesian analysis of homogeneity studies in the production of reference materials

Contact Info

Product

Resources

About