Demystifying EQA statistics and reports

Abstract

Reports act as an important feedback tool in External Quality Assessment (EQA). Their main role is to score laboratories for their performance in an EQA round. The most common scores that apply to quantitative data are Q- and Z-scores. To calculate these scores, EQA providers need to have an assigned value and standard deviation for the sample. Both assigned values and standard deviations can be derived chemically or statistically. When derived statistically, different anomalies against the normal distribution of the data have to be handled. Various procedures for evaluating laboratories are able to handle these anomalies. Formal tests and graphical representation techniques are discussed and suggestions are given to help choosing between the different evaluations techniques. In order to obtain reliable estimates for calculating performance scores, a satisfactory number of data is needed. There is no general agreement about the minimal number that is needed. A solution for very small numbers is proposed by changing the limits of evaluation. Apart from analyte- and sample-specific laboratory evaluation, supplementary information can be obtained by combining results for different analytes and samples. Various techniques are overviewed. It is shown that combining results leads to supplementary information, not only for quantitative, but also for qualitative and semi-quantitative analytes.

Clinical evaluation of commercial automated SARS-CoV-2 immunoassays

“…The SARS-CoV-2 IgM and IgG assays from Shenzhen YHLO Biotech (cut-off value: 10 AU/mL) and Roche Diagnostics (cut-off value: 1 COI/QE) were compared. The data were standardized using the Z-score calculation with Log (1) as cut-off value ( Coucke and Soumali, 2017 ). QRT-PCR-confirmed COVID-19 sera (n = 59) and prepandemic control sera (n = 50) were used.…”

“…For normalization of data, the Z-score was calculated – i.e., the difference between the measured value and the assigned value, corrected for the variability ( Coucke and Soumali, 2017 ).…”

Quality Management Systems for Laboratories and External Quality Assurance Programs

“…In case of nonregulatory EQA participation, the laboratories should decide the proper limits for the proposed objective. When the acceptance limit is defined as the "fitness for purpose", such purpose must be specified based on external requirements [15].…”

“…Independent of sample commutability, previous analysis tools are not valid for semiquantitative measures or measures reported on a discontinuous scale or where dichotomous results are provided for a continuous parameter [15].…”

Schemes and Performance Evaluation Criteria of Korean Association of External Quality Assessment (KEQAS) for Improving Laboratory Testing

“…Maximum tolerance limits can be statistically determined (e.g., using ± 3 SDIs or Z-scores) or established as fixed percentages or amounts (e.g., ± 15% of the target value or ± 10 mg/dL) [ 14 ]. As the SDI is a standardized value, it can be compared among all analytes [ 15 ]. However, the limitation of the statistical method is that when applying the SDI as a tolerance limit, the acceptable range for peer groups with larger SDs is larger than that for peer groups with smaller SDs.…”