External validation of an electrocardiography artificial intelligence-generated algorithm to detect left ventricular systolic function in a general cardiac clinic in Uganda

Abstract

Background Left ventricular systolic dysfunction (LVSD) is associated with increased morbidity and mortality. Although there are effective treatments for patients with LVSD to prevent mortality, heart failure and to improve symptoms, many patients remain undetected and untreated. We have recently derived a deep learning algorithm to detect LVSD using the electrocardiogram (ECG) which could have an important screening role, particularly in limited resources settings. We evaluated the accuracy of this algorithm for the first time in Africa in a sample of subjects attending a cardiology clinic. Methods We conducted a retrospective study in a general cardiac clinic in Uganda. Consecutive patients ≥18 years who had a digital ECG and echocardiogram done within two days of each other were included. We excluded patients with pacemakers or missing information regarding left ventricular ejection fraction (LVEF). Routine 10-second, twelve-lead surface rest ECG were performed using an Edan PC ECG Model SE-1515, Hamburg, Germany. The probability of LVSD was estimated with the Mayo Clinic artificial intelligence (AI) ECG algorithm. LVEF was calculated by the MMode (Teichholz method) using a Philips Ultrasound system, HD7XE, Bothel, Washington, USA. LVSD was defined as a LVEF≤35%. We assessed the overall diagnostic performance of the algorithm to identify LVSD in this population with the area under the receiver operating curve (AUC), and estimated sensitivity, specificity and accuracy using a pre-specified cut-off based on the probability for LVSD generated by the algorithm. We conducted secondary analyses using different LVEF cutoff values. Results We included 634 subjects, 32% (200) of whom had hypertension and 12% (77) clinical heart failure. Mean age was 57±18.8 years, 58% were women and the overall prevalence of LVSD was 4%. The AI-ECG had an AUC of 0.866 (see figure below), sensitivity 73.08%, specificity 91.10%, negative predictive value 98.75%, positive predictive value 26.03% and an accuracy of 90.96% using the original threshold. Using the optimal cutoff based on the AUCs, the sensitivity was 80.77% and specificity was 81.05% with a negative predictive value of 98.99%. The ROC for the detection of LVEF of 40% or below was 0.821. Conclusion The Mayo AI-ECG algorithm demonstrated good accuracy, sensitivity and specificity to detect LVSD in patients seen in a clinical setting in Uganda. This tool may facilitate the identification of people at a high risk for LVSD in settings with low resources. ROC Funding Acknowledgement Type of funding source: None

Deep learning of ECG waveforms for diagnosis of heart failure with a reduced left ventricular ejection fraction

“…Second, generalization of our results is limited, and should be cautiously interpreted, as the study population was drawn from a single hospital site in Korea. Further studies on a wider range of race and ethnicity are necessary, as done per the study conducted by the Mayo Clinic using an artificial intelligence-augmented electrocardiogram (AI-ECG) in the United States and Uganda 9 , 14 . Third, although most of the ECGs were matched to echocardiography within 24 h, some were performed within 30 days.…”

“…The use of ECG for LVSD diagnosis has been ongoing since 1996, from identification of simple abnormalities on ECG to the more recent development of artificial intelligence (AI) algorithms 5 , 7 – 15 . Various AI algorithms have been developed and performed based on different definitions of LVSD (e.g., ejection fraction (EF) < 35% 7 , 10 , 14 , < 40% 8 , 9 , 11 – 13 , or < 50% 12 ) and for distinct study populations 9 , 13 . Despite advancement in AI-based LVSD diagnosis, an AI algorithm to identify LVSD patients with an EF < 40% has not been validated in a clinical population of patients with symptomatic HF regardless of EF.…”

Adoption of Artificial Intelligence in the Ugandan Health Sector: a review of Literature

“…A total of 1766 patients were included in the study and they achieved areas under the receiver operating characteristic curve ranging from 83.1% for a single C5.0 ruleset to 88.5% for a random forest model, with the GLMnet at 87.5%. Mondo et al [28] employed an artificial intelligence-generated algorithm to detect the left ventricular systolic function in a general cardiac clinic in Uganda. By using an optimal cutoff based on the AUCs, they achieved a sensitivity of 80.77% and specificity of 81.05% with a negative predictive value of 98.99%.…”

Deep Learning of ECG Waveforms for Diagnosis of Heart Failure With a Reduced Left Ventricular Ejection Fraction

“…Second, generalization of our results is limited, and should be cautiously interpreted, as the study population was drawn from a single hospital site in Korea. Further studies on a wider range of race and ethnicity are necessary, as done per the study conducted by the Mayo Clinic using an arti cial intelligence-augmented electrocardiogram (AI-ECG) in the United States and Uganda 9,14 . Third, although most of the ECGs were matched to echocardiography within 24 h, some were performed within 30 days.…”

“…The use of ECG for LVSD diagnosis has been ongoing since 1996, progressing from identi cation of simple abnormalities on ECG to the more recent development of arti cial intelligence (AI) algorithms 5,7−15 . Various AI algorithms have been developed and performed based on different de nitions of LVSD (e.g., ejection fraction (EF) < 35% 7,10,14 , < 40% 8,9,11−13 , or < 50% 12 ) and for distinct study populations 9,13 . Despite advancement in AI-based LVSD diagnosis, an AI algorithm to identify LVSD among patients with an EF < 40% has not been validated in a clinical population of patients with symptomatic heart failure (HF).…”