Design standards for experimental and field studies to evaluate diagnostic accuracy of tests for infectious diseases in aquatic animals

Abstract

Design and reporting quality of diagnostic accuracy studies (DAS) are important metrics for assessing utility of tests used in animal and human health. Following standards for designing DAS will assist in appropriate test selection for specific testing purposes and minimize the risk of reporting biased sensitivity and specificity estimates. To examine the benefits of recommending standards, design information from published DAS literature was assessed for 10 finfish, seven mollusc, nine crustacean and two amphibian diseases listed in the 2017 OIE Manual of Diagnostic Tests for Aquatic Animals. Of the 56 DAS identified, 41 were based on field testing, eight on experimental challenge studies and seven on both. Also, we adapted human and terrestrial-animal standards and guidelines for DAS structure for use in aquatic animal diagnostic research. Through this process, we identified and addressed important metrics for consideration at the design phase: study purpose, targeted disease state, selection of appropriate samples and specimens, laboratory analytical methods, statistical methods and data interpretation. These recommended design standards for DAS are presented as a checklist including risk-of-failure points and actions to mitigate bias at each critical step. Adherence to standards when designing DAS will also facilitate future systematic review and meta-analyses of DAS research literature.

To pool or not to pool? Guidelines for pooling samples for use in surveillance testing of infectious diseases in aquatic animals

“…Both Gardner et al () and Laurin et al () discuss some of these factors in more detail at the DSe level. In the following subsections, we discuss the broad categories of these guidelines in more detail and provide examples of impacts on interpretation of DSe for pooled versus individual samples, highlighting areas that could lead to biased or incomplete interpretation of diagnostic accuracy estimates, and suggesting either mitigation strategies, statistical adjustments or the direction of any inherent bias that might affect interpretation and reporting of PSe results.…”

“…DFO, ; Tavornpanich et al, ). Therefore, transparent reporting (see Gardner et al, , and Laurin et al, ) of standard operating procedures (SOP) for pooling will help facilitate meta‐analyses and comparative studies.…”

“…Figure 2, it is critical that researchers clearly answer the following questions (in italics) prior to using pooled samples, as PSe will be affected by variations in these factors. Both Gardner et al (2016) and Laurin et al (2018) discuss some of these factors in more detail at the DSe level. In the following subsections, we discuss the broad categories of these guidelines in more detail and provide examples of impacts on interpretation of DSe for pooled versus individual samples, highlighting areas that could lead to biased or incomplete interpretation of diagnostic accuracy estimates, and suggesting either mitigation strategies, statistical F I G U R E 2 Decision tree of recommended guidelines for using pooling techniques for diagnostic test evaluation and surveillance studies of infectious diseases in aquatic animals.…”

“…Corsin et al (2015) recommend that pooled testing procedures follow 'accepted, statistically based methodologies, which must be fully documented, including published references', as in the studies by Ogut and Altuntas (2014) DFO, 2011;Tavornpanich et al, 2004). Therefore, transparent reporting (see Gardner et al, 2016, andLaurin et al, 2018) of standard operating procedures (SOP) for pooling will help facilitate metaanalyses and comparative studies.…”

“…However, if pooling is done prior to homogenization (e.g. Hall et al, 2013Hall et al, , 2014aJansen et al, 2010) The effect of homogenization, inhibition, efficiency and tissue sampled (analyte concentration and amount) on PSe interpretation can vary, dependent on the test's cut-off (see Laurin et al, 2018, for details on cut-off thresholds in DAS). In low-analyte concentration situations, dilution and inhibition effects due to pooling can easily lead to a false-negative result, which can be a costly consequence in disease freedom certification and import/export situations.…”