“…This is especially true for state-of-the-art deep learning models. As a result, most studies have not used explainability (Zhang et al, 2011;Kwon et al, 2018;Niroshana et al, 2019;Phan et al, 2019;Wang et al, 2020;Li et al, 2021), which is concerning because transparency is increasingly required to assist with model development and physician decision making (Sullivan and Schweikart, 2019). As such, more multimodal explainability methods need to be developed (Lin et al, 2019;Mellem et al, 2020;Ellis et al, 2021a,b,c,d).…”