Critique of the Guide to the Expression of Uncertainty in Measurement Method of Estimating and Reporting Uncertainty in Diagnostic Assays

Abstract

Background:The Guide to the Expression of Uncertainty in Measurement (GUM) provides instructions for constructing uncertainty intervals for a measurement. This method is usually reserved for reference materials, but GUM has been recently proposed as a way to express uncertainty for commercial diagnostic assays. Methods: Using the official GUM standard and published applications of GUM to commercial diagnostic assays, I undertook an analysis to evaluate whether applying GUM to commercial diagnostic assays is warranted. Results: Certain important assays, such as troponin I, would not be candidates for GUM because troponin I is not a well-defined physical quantity. Unlike definitive methods, in which efforts are taken to detect and eliminate all systematic error sources, commercial assays often trade off features such as ease of use and cost with accuracy and allow systematic errors to be present as long as the overall accuracy meets the medical need goal. Laboratories are hindered in preparing GUM models because the knowledge required to specify some systematic errors is often available only to manufacturers. Some non-GUM methods to estimate uncertainty rely on observed data, which include both known and unknown sources of error. The occurrence of large, unknown errors for assays in routine use (e.g., outliers) is not unusual because diagnostic assays must be chemically specific in the presence of thousands of potentially interfering substances. There is no provision in GUM to deal with unexplained outliers, which may lead to uncertainty intervals that are not wide enough. Some clinicians assume that diagnostic assay results have little uncertainty. This situation may be made worse by including an uncertainty interval, which implies certification. Conclusions: Evaluations for accuracy (total analytical error) based on describing the distribution of result differences between commercial assays and reference methods indicate that some assays have a few results with large differences (e.g., outliers). This leads to a wide accuracy interval (total analytical error limits). It is unlikely that GUM would be able to predict these wide intervals, especially because there is little or no provision for outlier treatment in GUM. Presenting too narrow GUM uncertainty intervals to clinicians would be misleading. The modeling used by practitioners of the GUM method is potentially useful in improving quality, but commercial diagnostic assays are not ready for GUM uncertainty statements. © 2003 American Association for Clinical ChemistryAccuracy, the closeness in agreement between a test result and an accepted reference method, is a concept well known to clinical chemists. Inaccuracy, the lack of closeness in agreement (also called total analytical error), is caused by random and systematic errors. (3, 4 ). Kristiansen (5 ) described how GUM applies to diagnostic assays. Here I will describe problems with the GUM method with respect to its use for routine diagnostic assays. Kallner (6 ) and the NIST web site (7 ) provide excelle...

Within the context of good clinical laboratory practices evaluation of measurement uncertainty of Thyroid Stimulating Hormone and Prostate Specific Antigen parameters

“…Sıklıkla rapor edilen hatalar göstermiştir ki, tanısal ölçümlerde interferans hataları önemli bir yer tutmaktadır. Bu hatalar kabul edilebilir belirsizlik aralığını aştığında çok önemlidir [6].…”

“…Krouwer'a göre ticari kitler GUM'un belirsizlik standartlarına uygun değildir [6]. Ayrıca Krouwer, yöntemin doğruluğu değerlendirilirken referans yöntem ile ticari ölçümler karşılaştırıldığında bazı sonuçların dağılımın Sonuç olarak klinik laboratuarların test sonuçlarının belirsizliğini belirleyecek prosedürlere sahip olması uluslararası standartlar gereğidir [3].…”

“…Krouwer, GUM'un çok dar bir aralığı alması sonucu elde edilen belirsizlik ile klinisyenlerin birçok vakayı atlayabileceği görüşündedir. Krouwer'a göre, ölçümün doğrulu-ğuna katkıda bulunan her hata kaynağının detaylarını ele alan bir matematiksel model yaratılmalıdır [6].…”

Measurement Uncertainty in Forensic Toxicology: Its Estimation, Reporting and Interpretation

“…2007, Kacker,et.al. 2010, Krouwer, 2003, Kristiansen, 2003 NIST Measurement Results: (NIST, 1994) This document is brief but includes the key concepts and definitions. There are very few illustrated examples.…”

Evaluation of Assigned-Value Uncertainty for Complex Calibrator Value Assignment Processes: A Prealbumin Example

“…Aside from fulfilling In Vitro Diagnostics Medical Devices compliance requirements, the clinical utility of estimating and reporting uncertainty in diagnostic assays has been debated (5,6 ). We found Monte Carlo simulations to be useful in the design and optimization of the value-assignment process, because this method allows for changing simulated process parameters such as testing replication and number of instruments.…”