Assessment and validation of a novel fast fully automated artificial intelligence left ventricular ejection fraction quantification software

Abstract

Background: Quantification of left ventricular ejection fraction (LVEF) by transthoracic echocardiography (TTE) is operator-dependent, time-consuming, and error-prone. LVivoEF by DIA is a new artificial intelligence (AI) software, which displays the tracking of endocardial borders and rapidly quantifies LVEF. We sought to assess the accuracy of LVivoEF compared to cardiac magnetic resonance imaging (cMRI) as the reference standard and to compare LVivoEF to the standard-of-care physician-measured LVEF (MD-EF) including studies with ultrasound enhancing agents (UEAs). Methods: In 273 consecutive patients, we compared MD-EF and AI-derived LVEF to cMRI. AI-derived LVEF was obtained from a non-UEA four-chamber view without manual correction. Thirty-one patients were excluded: 25 had interval interventions or incomplete TTE or cMRI studies and six had uninterpretable non-UEA apical views.Results: In the 242 subjects, the correlation between AI and cMRI was r = .890, similar to MD-EF and cMRI with r = .891 (p = 0.48). Of the 126 studies performed with UEAs, the correlation of AI using the unenhanced four-chamber view was r = .89, similar to MD-EF with r = .90. In the 116 unenhanced studies, AI correlation was r = .87, similar to MD-EF with r = .84. From Bland-Altman analysis, LVivoEF underreported the LVEF with a bias of 3.63 ± 7.40% EF points compared to cMRI while MD-EF to cMRI had a bias of .33 ± 7.52% (p = 0.80). Conclusions:Compared to cMRI, LVivoEF can accurately quantify LVEF from a standard apical four-chamber view without manual correction. Thus, LVivoEF has the ability to improve and expedite LVEF quantification.

Single-Site Experience with an Automated Artificial Intelligence Application for Left Ventricular Ejection Fraction Measurement in Echocardiography

“…Although our experienced Echo examiners were trained and received regular briefings in order to overcome the technical obstacles in image acquisition, the measurement bias of LVEF between human operators and AI detected by our work cannot be ignored ( Figure 2 ). Interestingly, the LVEF bias of 4.5% is congruent with the result of a previous study of this particular AI [ 9 ] and other recently published machine /deep learning-based approaches for EF quantification [ 4 , 6 , 7 , 8 ]. The LV volumes measured by these applications are greater than those measured by human operators using the MBS method, a finding we also observed to a similar extent in our study ( Table 2 ).…”

“…In other words, the smaller the ESV was determined to be by the MBS method, the greater the bias of the LVEF value compared to that of AI measurements. This suggests some subjectivity and inconsistency in endocardial tracing by human operators, as pointed out in previous studies, regardless of the imaging technique used [ 9 , 16 , 17 ].…”

“…Lastly, one could question why we did not primarily use identical echocardiographic views for the measurement of LV function, a common method in AI validation studies [ 9 , 21 , 25 ]. This was intentional and directly related to our goal of testing this AI not only as a mere replacement for the conventional Echo methodology in daily practice, but also as a “second pair of eyes”.…”

“…Test–retest variability was assessed by the coefficient of variability (COV), defined as the absolute difference of the corresponding pair of repeated measurements in a percentage of their mean for each patient and then averaged over the study group. For the reproducibility analysis, we calculated a sample size of 30 with an ICC of 0.91 [ 9 ] and a null hypothesis value of ICC = 0.79 with power 0.80 and alpha 0.05 (R package “ICC. Sample.…”

“…Recently, a commercially available and ready-to-use AI-based application for fully automated LV classification and quantification without human interaction (LVivo Seamless TM , DIA Imaging Analysis, Beer-Sheva, Israel) has been developed to address exactly these issues. Its accuracy has been validated with CMR imaging, but only on selected patients [ 9 ]. To address this issue, we sought to prospectively evaluate the applicability and feasibility of this AI-based application in a real-world clinical setting on a large, unselected cohort of patients.…”