The Ventricular Volume Variability Study of the Pediatric Heart Network: Study Design and Impact of Beat Averaging and Variable Type on the Reproducibility of Echocardiographic Measurements in Children with Chronic Dilated Cardiomyopathy

Abstract

Clinical trials often rely on echocardiographic measures of left ventricular (LV) size and function as surrogate end-points. However, the quantitative impact of factors that affect reproducibility of these measures is unknown. To address this issue, the NHLBI-funded Pediatric Heart Network designed a longitudinal observational study of children with known or suspected dilated cardiomyopathy (DCM) aged 0–22 years from 8 pediatric clinical centers. Methods Clinical data were collected together with 150 echocardiographic indices of LV size and function. Separate observers performed duplicate echocardiographic imaging. Multiple observers performed measurements from three cardiac cycles to enable assessment of intra and interobserver variability. We studied the impact of beat averaging (BA), observer type (local vs. core) and variable type (areas, calculations, dimensions, slopes, time intervals and velocities) on measurement reproducibility. The outcome measure was %error (100 × difference/mean) Results Of 173 enrolled subjects, 131 met criteria for DCM. BA, variable type and observer type all impacted %error (p<0.0001). Core inter-observer %error (median 11.4, 10.2 and 9.3% for 1-, 2- and 3-BA, respectively) was approximately twice the intra-observer %error (median 6.3, 4.9 and 4.2% for 1-, 2- and 3-BA, respectively). Slopes and calculated variables exhibited high %error despite BA. Chamber dimensions, areas, velocities and time intervals exhibited low %error. Conclusions This comprehensive evaluation of quantitative echocardiographic methods will provide a valuable resource for design of future pediatric studies. BA and a single core lab observer improve reproducibility of echo measurements in children with DCM. Certain measurements are highly reproducible, while others, despite BA, are poorly reproducible.

Quality Improvement Implementation: Improving Reproducibility in the Echocardiography Laboratory

“…Commonly used methods have included the Pearson correlation coefficient, percentage error, intraclass correlation coefficient, and k statistic. [21][22][23] The limitations of these statistical methods are that they are influenced by population characteristics, lack data on individual readers, and are dependent on the range of data in the analyzed sample. 18,19,24 For example, the intraclass correlation coefficient for the initial evaluation of LVEDV in this study was 0.88, which is seemingly a good correlation for the group but does not provide any information about individual variability such as that seen with reader A, nor does it indicate the source of interpretative error.…”

“…Prior studies on reproducibility have provided valuable insight about the inconsistency of echocardiographic results in research and clinical practice, but these studies were generally descriptive evaluations of reproducibility that lacked recommendations or effective interventions to minimize variability once it was identified. 11,21 These studies limited the evaluation of interobserver variability and echocardiographic reproducibility to a few parameters (most commonly LV function) or a single disease entity. [11][12][13]16,21 In contrast, we included eight clinically important echocardiographic parameters, as well as a spectrum of cardiac pathologies commonly encountered in clinical practice to make certain that this method was broadly applicable.…”

“…11,21 These studies limited the evaluation of interobserver variability and echocardiographic reproducibility to a few parameters (most commonly LV function) or a single disease entity. [11][12][13]16,21 In contrast, we included eight clinically important echocardiographic parameters, as well as a spectrum of cardiac pathologies commonly encountered in clinical practice to make certain that this method was broadly applicable. In addition, this approach begins to address the longstanding issue of how to sufficiently balance precision (smaller differences are more precise) and feasibility (larger differences are easier to achieve) by setting acceptable differences for each parameter.…”

Longitudinal systolic strain of the bilayered ventricular septum during the first 72 hours of life in preterm infants

“…Care was taken to keep the ultrasound beam aligned parallel to the VS and RV free wall. Images were graded according to the scoring system [20], and only images graded as excellent or good were included in the following analysis. Regions of interest were manually positioned on the right and left sides of the VS separated by a bright line and the RV free wall, both at the middle part of the wall (Figs.…”

Quality Improvement in Noninvasive Imaging: Present and Future Initiatives

“…Many reports have been published comparing echo diagnoses with data from other modalities (such as cardiac magnetic resonance imaging, computed tomography imaging, and cardiac catheterization) as well as fi ndings in the operating room during surgery [ 19 -22 ]. More recently, other publications have evaluated multi-acquisition, intra-observer, and inter-observer variability for echo measurements and have shown that some measurements are more prone to poor repeatability than others, even with multiple heart beat averaging [ 23 ]. The IAC has developed specifi c recommendations for quarterly correlation assessments (to compare echo fi ndings in specifi c instances with results from other modalities and from the operating room) and variability assessments (to evaluate the repeatability of specifi c measurements performed during an echo) [ 18 ].…”