Performance adjusted risks: a method to improve the quality of algorithm performance while allowing all to play

Evans, Mark I.; Cuckle, Howard

doi:10.1002/pd.2769

Cited by 8 publications

(10 citation statements)

References 24 publications

(33 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, we demonstrated that the actual performance of NT measurements in the United States has excessive variability, and the curve of measurements is skewed to the left, meaning that the likelihood of under-measurement is much greater than it should be [5,6,7,8]. Under-measurements result in improperly underestimating aneuploidy risks to patients and provide false reassurance to some.…”

Section: Discussionmentioning

confidence: 99%

“…One solution has been the development of a method (performance adjusted risks) to help improve the overall quality of risk determination while recognizing variability of the providers of NT measurements [7]. The more general issue is QC which has been a constant issue in laboratory and clinical medicine, but it is more easily quantified in the lab.…”

Section: Discussionmentioning

confidence: 99%

“…New methods have variable acceptance and performance capabilities [7,17,18]. Their introductioncan be divided into two distinct phases: development and diffusion.…”

Section: Discussionmentioning

confidence: 99%

“…It has been our experience over the last decade that the introduction of NT screening has been associated with a far higher level of inaccuracy and poor performance than the corresponding biochemical measures used in the determination of risk for aneuploidy [5,6,7,8]. …”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Quality of Nuchal Translucency Measurements Correlates with Broader Aspects of Program Rigor and Culture of Excellence

Evans

Krantz²,

Hallahan³

et al. 2013

Fetal Diagn Ther

Self Cite

View full text Add to dashboard Cite

Introduction: To determine if nuchal translucency (NT) quality correlates with the extent to which clinics vary in rigor and quality control. Methods: We correlated NT performance quality (bias and precision) of 246,000 patients with two alternative measures of clinic culture - % of cases for whom nasal bone (NB) measurements were performed and % of requisitions correctly filled for race-ethnicity and weight. Results: When requisition errors occurred in <5% of cases, the average MoM (multiple of the median) was 0.97. When >5% (33%), the curve lowered to 0.93 MoM (p < 0.001) with both bias and precision of measurements impaired. Likewise, for centers with NB >90%, MoM was 0.99 compared to those <10% at 0.93 (p < 0.001). Precision and bias were highly correlated (p < 0.001). Conclusions: Rigor in NT measurements has improved, but the discussion has been confined to individuals. Progressive educational and remediation strategies need to expand to a second dimension - clinics themselves. Cross-clinic variation in NT quality exists independent of individual variation in NT quality, and two divergent indices of program rigor are associated with NT quality. Quality control must be program wide, and to effect continued improvement in the quality of NT results across time, the cultures of clinics must become a target for intervention.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

“…New methods have variable acceptance and performance capabilities [7,17,18]. Their introductioncan be divided into two distinct phases: development and diffusion.…”

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Quality of Nuchal Translucency Measurements Correlates with Broader Aspects of Program Rigor and Culture of Excellence

Evans

Krantz²,

Hallahan³

et al. 2013

Fetal Diagn Ther

Self Cite

View full text Add to dashboard Cite

show abstract

“…Kagan et al [7] demonstrated that an underestimation in NT measurement reduced the detection rate by up to 5%. Evans and Cuckle [19] demonstrated that performance-adjusted risk allows for performance differences among sonographers reporting NT measurements.…”

Section: Discussionmentioning

confidence: 99%

Impact of Including or Removing Nuchal Translucency Measurement on the Detection and False-Positive Rates of First-Trimester Down Syndrome Screening

Spaggiari

Czerkiewicz²,

Sault³

et al. 2015

Fetal Diagn Ther

View full text Add to dashboard Cite

Introduction: First-trimester Down syndrome (DS) screening combining maternal age, serum markers (pregnancy-associated plasma protein-A and beta-human chorionic gonadotropin) and nuchal translucency (NT) gives an 85% detection rate for a 5% false-positive rate. These results largely depend on quality assessment of biochemical markers and of NT. In routine practice, despite an ultrasound quality control organization, NT images can be considered inadequate. The aim of the study was to evaluate the consequences for risk calculation when NT measurement is not taken into account. Material and Method: Comparison of detection and false-positive rates of first-trimester DS screening (PerkinElmer, Turku, Finland), with and without NT, based on a retrospective study of 117,126 patients including 274 trisomy 21-affected fetuses. NT was measured by more than 3,000 certified sonographers. Results: There was no significant difference in detection rates between the two strategies including or excluding NT measurement (86.7 vs. 81.8%). However, there was a significant difference in the false-positive rates (2.23 vs. 9.97%, p < 0.001). Discussion: Sonographers should be aware that removing NT from combined first-trimester screening would result in a 5-fold increase in false-positive rate to maintain the expected detection rates. This should be an incentive for maintaining quality in NT measurement.

show abstract

Nuchal Translucency Quality Review (NTQR) program: first one and half million results

Cuckle

Platt

Thornburg

et al. 2015

Ultrasound in Obstet & Gyne

View full text Add to dashboard Cite

show abstract

Performance adjusted risks: a method to improve the quality of algorithm performance while allowing all to play

Abstract: PAR recognizes that not all providers are equal, and perfection is unrealistic. Using this approach all 'can play' while patients are protected from poor performance.

Cited by 8 publications

References 24 publications

Quality of Nuchal Translucency Measurements Correlates with Broader Aspects of Program Rigor and Culture of Excellence

Quality of Nuchal Translucency Measurements Correlates with Broader Aspects of Program Rigor and Culture of Excellence

Impact of Including or Removing Nuchal Translucency Measurement on the Detection and False-Positive Rates of First-Trimester Down Syndrome Screening

Nuchal Translucency Quality Review (NTQR) program: first one and half million results

Contact Info

Product

Resources

About