Systematic error and cognitive bias in obstetric ultrasound

Abstract

This issue of the Journal hosts an important systematic review assessing methodological aspects of studies constructing Doppler reference values for two clinically important vessels, the umbilical and fetal middle cerebral arteries. In this review, Oros et al. 1 developed a 24-item scale with which to assess the methodology of Doppler reference studies over three domains, namely, study design, statistics and reporting methods. Most of the included studies performed moderately for most of the items, which was reflected in the 51.4% overall quality score. Common methodological limitations included retrospective collection of data from fetuses which were scanned for clinical rather than for research reasons, performance of the measurements by a single operator, non-blinded measurements and lack of monitoring of ultrasound quality. A consequence of the multiple limitations, as well as of the methodological heterogeneity of the Doppler studies, is that there was significant variation in normal ranges, even across high-quality studies, which may undermine the clinical applicability of these ranges. Having detected several problematic areas, the authors propose a set of criteria aiming to limit bias in future studies 1 .Although ultrasound examination is an integral part of current obstetric care and a major tool for making clinical decisions, the systematic errors affecting its efficacy have been assessed only marginally. The literature on obstetric ultrasound is abundant in studies assessing its repeatability and agreement, but it is alarmingly poor when it comes to sources of error. Error in obstetric ultrasound can occur at the level of studies, in the form of bias, and at the level of the individual examiner, in the form of mistaken interpretation of the clinical evidence, leading to mistaken diagnosis and, possibly, mismanagement of the patient. Although these two levels commonly overlap and interact, we will discuss them separately for the sake of clarity. Bias at level of studiesAll studies, especially non-randomized and retrospective trials, are susceptible to various sources of bias. In this Editorial, we will focus on specific types of bias that are particularly pertinent to obstetric ultrasound studies. Expectation biasOros et al. 1 reported that Doppler measurements were blinded in only one of 38 Doppler reference studies, which renders their results susceptible to expected-value bias or expectation bias. In this scenario, clinical examiners affected by expectation bias subconsciously regress their measurements closer to the expected range when the latter is known at the time of the examination. As a result, the dispersion of the measurements decreases. Although there is no quantitative information available on the magnitude of this effect, it is reasonable to hypothesize that normal ranges constructed using data acquired retrospectively will be relatively narrow, and examiners using them will then modify their measurements to conform to these narrow ranges, for as long as this remains technically feasible....

Expected‐value bias in routine third‐trimester growth scans

“…Blinding makes it difficult to bias results intentionally or unintentionally and so helps to ensure the credibility of measurements. Recently, a review of systematic error and cognitive bias in obstetric ultrasound suggested that expectation bias is pertinent to obstetric ultrasound studies.…”

Expected‐value bias in mid‐trimester preterm birth screening

“…Systematic observer error may occur if the observer has a preconceived idea of what a measurement ought to be, leading to adjustment of the reading by the observer 13 . Systematic errors affecting the efficacy of obstetric ultrasound have not been assessed extensively 14 .…”

“…Systematic observer error may occur if the observer has a preconceived idea of what a measurement ought to be, leading to adjustment of the reading by the observer 13 . Systematic errors affecting the efficacy of obstetric ultrasound have not been assessed extensively 14 . Recently, expected‐value bias has been reported to affect biometry in clinical practice 13 and Doppler assessment for construction of reference ranges 15 .…”

Late presentation of a heterotopic pregnancy at 19 weeks of gestation leading to maternal collapse: A case report

“…A high index of suspicion is required to diagnose HP in a general setting, where rates are lower. HPs may frequently be missed on imaging due to both anchoring bias (whereby the clinician clings to the initial diagnostic impression and fails to adjust their impression in light of subsequent information) and satisfaction of search bias (whereby the clinician stops searching for abnormalities once a diagnosis perceived as being likely has been reached) [ 5 ].…”