Guidelines for improving statistical analyses of validation datasets for plant pest diagnostic tests

Massart, Sébastien; Lebas, B. S. M.; Chabirand, Aude; Chappé, Anne‐Marie; Dreo, T.; Faggioli, F.; Harrison, Catherine; Macarthur, Roy; Mehle, Natasha; Mezzalama, Mónica; Petter, F.; Ravnikar, Maja; Renvoisé, Jean‐Philippe; Spadaro, D.; Tomassoli, L.; Tomlinson, J. A.; Trontin, Charlotte; Vlugt, René van der; Vučurović, Ana; Weekes, Rebecca; Brostaux, Yves

doi:10.1111/epp.12862

Cited by 6 publications

(11 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If conclusions are based on the results of <10 laboratories, they should be presented with caution and appropriate warnings. This is because the resulting performance characteristics will be subject to an increased uncertainty with possible incorrect estimations of the confidence intervals due to the limited number of laboratories (Brotaux et al, 2021; Massart et al, 2022).…”

Section: Technical Requirementsmentioning

confidence: 99%

“…T A B L E 4 Example of a panel of 25 samples for the evaluation of analytical sensitivity, diagnostic sensitivity, diagnostic specificity, repeatability and reproducibility in a TPS. This panel of samples, proposed in the framework of the VALITEST project, was considered as a good balance between statistical power and practicality for TPS organizers (see Brotaux et al, 2021;Massart et al, 2022;Vučurovič et al, 2022).…”

Section: Statistical Analysis Of Homogeneity Testingmentioning

confidence: 99%

“…If the number of samples vary between laboratories and tests, it is particularly important to include confidence intervals in the analysis. Confidence intervals for accuracy, diagnostic sensitivity and diagnostic specificity can be calculated using the Agresti-Coull method (Agresti & Coull, 1998;Brotaux et al, 2021;Massart et al, 2009a, b Massart et al, 2022.…”

Section: R E F E R E Nc E Smentioning

confidence: 99%

“…Details on how to calculate accordance and concordance can be found in Langton et al (2002), Brotaux et al, 2021, Massart et al (2022.…”

Section: Intra-and Interlaboratory Variationsmentioning

confidence: 99%

“…Methods based on calculations of probability of detection (POD) can be used to compare the analytical sensitivity of different tests, in particular for TPS (Wehling et al, 2011). A binomial generalized linear model (bGLM) with a logit link function (log[p/(1-p] where p is the probability of detection) was recommended as POD model in the framework of VALITEST (Brotaux et al, 2021;Massart et al, 2022). This statistical method does not require any assumption on the number of technical and/ or biological replicates so that those numbers can vary between samples and/or laboratories.…”

Section: Analytical Sensitivitymentioning

confidence: 99%

See 4 more Smart Citations

PM 7/122 (2) Guidelines for the organization of interlaboratory comparisons by plant pest diagnostic laboratories

2022

EPPO Bulletin

View full text Add to dashboard Cite

Section: Technical Requirementsmentioning

confidence: 99%

Section: Statistical Analysis Of Homogeneity Testingmentioning

confidence: 99%

Section: R E F E R E Nc E Smentioning

confidence: 99%

“…Details on how to calculate accordance and concordance can be found in Langton et al (2002), Brotaux et al, 2021, Massart et al (2022.…”

Section: Intra-and Interlaboratory Variationsmentioning

confidence: 99%

Section: Analytical Sensitivitymentioning

confidence: 99%

See 3 more Smart Citations

PM 7/122 (2) Guidelines for the organization of interlaboratory comparisons by plant pest diagnostic laboratories

2022

EPPO Bulletin

View full text Add to dashboard Cite

Description of the Process of TPS Organisation

Anthoine

Brittain

Chappé

et al. 2022

Plant Pathology in the 21st Century

View full text Add to dashboard Cite

The organisation of a test performance study (TPS) involves different steps that are mostly sequential, but some may be conducted simultaneously. This chapter details the following: the steps regarding the selection of the tests to be validated; the selection of the laboratories to participate in the TPS; the preparation of the materials and the dispatch of the samples; and the completion of the TPS (including the collection and analysis of the TPS results). The reader will be able to get the detailed information on how to define and plan timeline of the TPS, the appropriate number of samples (including replicates) and of laboratories that should be included in the TPS to ensure an appropriate statistical analysis, and how to perform basic analyses of the obtained data. In addition, this chapter covers the most important critical points which can endanger successful TPS organization providing the future TPS organisers in the field of plant health (but also in other similar fields) with the possibility to identify them in advance and carry-out successful TPS.

show abstract

Cont-ID: Detection of samples cross-contamination in viral metagenomic data

Rollin

Rong

Massart

2023

Preprint

View full text Add to dashboard Cite

Background: High Throughput sequencing (HTS) technologies completed by the bioinformatic analysis of the generated data are becoming an important detection technique for virus diagnostics. They have the potential to replace or complement the current PCR-based methods thanks to their improved inclusivity and analytical sensitivity, as well as their overall good repeatability and reproducibility. Cross-contamination is a well-known phenomenon in molecular diagnostics and corresponds to the exchange of genetic material between samples. Cross-contamination management was a key drawback during the development of PCR-based detection and is now adequately monitored in routine diagnostics. HTS technologies are facing similar difficulties due to their very high analytical sensitivity. As a single viral read could be detected in millions of sequencing reads, it is mandatory to fix a detection threshold that will be influenced by cross-contamination. Cross-contamination monitoring should therefore be a priority when detecting viruses by HTS technologies. Results: We present Cont-ID, a bioinformatic tool designed to check for cross-contamination by analysing the relative abundance of virus sequencing reads identified in sequence metagenomic datasets and their duplication between samples. It can be applied when the samples in a sequencing batch have been processed in parallel in the laboratory and with at least one external alien control. Using 273 real datasets, including 68 virus species from different hosts (fruit tree, plant, human) and several library preparation protocols (Ribodepleted total RNA, small RNA and double stranded RNA), we demonstrated that Cont-ID classifies with high accuracy (91%) viral species detection into (true) infection or (cross) contamination. This classification raises confidence in the detection and facilitates the downstream interpretation and confirmation of the results by prioritising the virus detections that should be confirmed. Conclusions: Cross-contamination between samples when detecting viruses using HTS can be monitored and highlighted by Cont-ID (provided an alien control is present). Cont-ID is based on a flexible methodology relying on the output of bioinformatics analyses of the sequencing reads and considering the contamination pattern specific to each batch of samples. The Cont-ID method is adaptable so that each laboratory can optimise it before its validation and routine use.

show abstract

Guidelines for improving statistical analyses of validation datasets for plant pest diagnostic tests

Abstract: Appropriate statistical analysis of the validation data for diagnostic tests facilitates the evaluation of the performance criteria and increases the confidence

Cited by 6 publications

References 24 publications

PM 7/122 (2) Guidelines for the organization of interlaboratory comparisons by plant pest diagnostic laboratories

PM 7/122 (2) Guidelines for the organization of interlaboratory comparisons by plant pest diagnostic laboratories

Description of the Process of TPS Organisation

Cont-ID: Detection of samples cross-contamination in viral metagenomic data

Contact Info

Product

Resources

About