Comparative analysis of Attention-based CNN andViT for processing ECG image data in joint fusion

Abstract

Percutaneous Coronary Intervention (PCI) is a widely utilized standard ofcare and efficacious intervention in the management of coronary artery disease,improving prognosis and symptoms for many patients. Prediction of post-PCIoutcomes contributes to effective patient management and quality improvementinitiatives in healthcare. Our study focuses on comparative performance analysisfor state-of-the-art vision transformer (ViT) and convolutional neural network(CNN) on multimodal data analysis in a joint fusion framework. We integrateimages of electrocardiogram (ECG) data and tabular electronic health records(EHR) to predict 3 clinically relevant post-PCI (6 months) endpoints - heart failure hospitalization, all-cause mortality, and ischemic stroke. To design a comparativemodel for ViT, we proposed a new joint fusion architecture that consistsof a convolutional neural network (CNN) with a convolutional block attentionmodule (CBAM). The learned representations are combined at an intermediatelayer followed by processing these representations using a fully connectedlayer. The proposed model demonstrates excellent performance with the highestAUROC scores of 0.849, 0.913, and 0.794 for predicting heart failure hospitalization,all-cause mortality, and stroke, respectively. Surpassing the baseline EHRmodel and ViT, the proposed CNN+CBAM fusion model showcases superiorpredictive capabilities for heart failure prediction.