Lack of grading agreement among international hemostasis external quality assessment programs

Abstract

Laboratory quality programs rely on internal quality control and external quality assessment (EQA). EQA programs provide unknown specimens for the laboratory to test. The laboratory's result is compared with other (peer) laboratories performing the same test. EQA programs assign target values using a variety of methods statistical tools and performance assessment of ‘pass’ or ‘fail’ is made. EQA provider members of the international organization, external quality assurance in thrombosis and hemostasis, took part in a study to compare outcome of performance analysis using the same data set of laboratory results. Eleven EQA organizations using eight different analytical approaches participated. Data for a normal and prolonged activated partial thromboplastin time (aPTT) and a normal and reduced factor VIII (FVIII) from 218 laboratories were sent to the EQA providers who analyzed the data set using their method of evaluation for aPTT and FVIII, determining the performance for each laboratory record in the data set. Providers also summarized their statistical approach to assignment of target values and laboratory performance. Each laboratory record in the data set was graded pass/fail by all EQA providers for each of the four analytes. There was a lack of agreement of pass/fail grading among EQA programs. Discordance in the grading was 17.9 and 11% of normal and prolonged aPTT results, respectively, and 20.2 and 17.4% of normal and reduced FVIII results, respectively. All EQA programs in this study employed statistical methods compliant with the International Standardization Organization (ISO), ISO 13528, yet the evaluation of laboratory results for all four analytes showed remarkable grading discordance.

Reporting of activated partial thromboplastin time (aPTT): Could we achieve better comparability of the results?

“…As aPTT(s) reference interval for the same aPTT reagent differ for different analytical systems, it is not uncommon for this limit to be exceeded when EQA results are compared for the same reagent group only. Reviewing the proficiency testing results for aPTT, obtained by other EQA providers, different approach for result assessment could be found ( 8 - 11 ). Different providers assess differently reported results (exclusively as seconds or as ratio, or both), and also apply different acceptable limits of the performance ( 8 , 9 ).…”

“…Reviewing the proficiency testing results for aPTT, obtained by other EQA providers, different approach for result assessment could be found ( 8 - 11 ). Different providers assess differently reported results (exclusively as seconds or as ratio, or both), and also apply different acceptable limits of the performance ( 8 , 9 ). Results are assessed according to system in use (reagent/coagulometer) and allowable limit is mainly set up in between 15 and 25% ( 8 , 9 ).…”

“…Different providers assess differently reported results (exclusively as seconds or as ratio, or both), and also apply different acceptable limits of the performance ( 8 , 9 ). Results are assessed according to system in use (reagent/coagulometer) and allowable limit is mainly set up in between 15 and 25% ( 8 , 9 ). The acceptable limit of 7% used by CROQALM originates from biological variability database and was set up in 2017 to assess performance of the aPTT results reported as both, seconds and ratio ( 10 ).…”

“…Reporting of the results in both ways was also recommended in recently published national guidelines ( 7 ). According to literature, different external quality assessment (EQA) providers assess performance of aPTT in seconds, ratio or in both ways, applying quite different acceptable limits of performance, ranging from 15 to 25% ( 8 , 9 ). Acceptable bias of 7% for aPTT result assessment, used by CROQALM from 2017, originates from biological variability database ( 10 , 11 ).…”

Harmonization of External Quality Assessment Schemes and their role – clinical chemistry and beyond

“…The problem of the evaluation criteria is very relevant, and their setting depends on the main scope of the program: if it has a regulatory impact, the criteria are looser because the scope is to identify the very poor performer; if it has an educational scope, the criteria are tighter because failing to meet them does not automatically imply sanctions but only a remedial action by the laboratory. This problem exists also outside clinical chemistry as indicated by Olson and coworkers for coagulation [28]. The situation may improve with the implementation on a larger scale of accreditation for EQA providers based on ISO 17043:2010 [29].…”