Approximate Explanations for Classification of Histopathology Patches

Sousa, Iam Palatnik de; Vellasco, Marley; Silva, Eduardo Costa da

doi:10.1007/978-3-030-65965-3_35

Cited by 3 publications

(7 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As an alternative to measure feature contribution, input region importance was also analyzed to reveal sub-region relevance to each prediction class. Image occlusion with blank subregions [109][110][111] and healthy-looking sub-regions 112 was used to find the most informative and relevant sub-regions for classification and detection tasks.…”

Section: In: Interpretabilitymentioning

confidence: 99%

Explainable medical imaging AI needs human-centered design: guidelines and evidence from a systematic review

et al. 2022

View full text Add to dashboard Cite

Transparency in Machine Learning (ML), often also referred to as interpretability or explainability, attempts to reveal the working mechanisms of complex models. From a human-centered design perspective, transparency is not a property of the ML model but an affordance, i.e., a relationship between algorithm and users. Thus, prototyping and user evaluations are critical to attaining solutions that afford transparency. Following human-centered design principles in highly specialized and high stakes domains, such as medical image analysis, is challenging due to the limited access to end users and the knowledge imbalance between those users and ML designers. To investigate the state of transparent ML in medical image analysis, we conducted a systematic review of the literature from 2012 to 2021 in PubMed, EMBASE, and Compendex databases. We identified 2508 records and 68 articles met the inclusion criteria. Current techniques in transparent ML are dominated by computational feasibility and barely consider end users, e.g. clinical stakeholders. Despite the different roles and knowledge of ML developers and end users, no study reported formative user research to inform the design and development of transparent ML models. Only a few studies validated transparency claims through empirical user evaluations. These shortcomings put contemporary research on transparent ML at risk of being incomprehensible to users, and thus, clinically irrelevant. To alleviate these shortcomings in forthcoming research, we introduce the INTRPRT guideline, a design directive for transparent ML systems in medical image analysis. The INTRPRT guideline suggests human-centered design principles, recommending formative user research as the first step to understand user needs and domain requirements. Following these guidelines increases the likelihood that the algorithms afford transparency and enable stakeholders to capitalize on the benefits of transparent ML.

show abstract

Section: In: Interpretabilitymentioning

confidence: 99%

Explainable medical imaging AI needs human-centered design: guidelines and evidence from a systematic review

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Alternatively, achieving transparency in an ML model by revealing its working mechanisms is widely hypothesized to invoke user trust in ML systems [97,16]. There have been approaches to provide transparency into decision making processes by incorporating prior knowledge directly into the model structure and/or inference process in hopes of invoking interpretability [3,18] or providing post-hoc explanations for black box models [24,54]. However, as we will highlight in detail through a systematic review, current approaches that aim at incorporating transparency to ML systems rely on developers' intuition on what may be transparent, rather than considering whether these mechanisms affect users' experience with the system and their ability to interpret ML model's outputs.…”

Section: Machine Learning For Healthcarementioning

confidence: 99%

“…As an alternative to measure feature contribution, input region importance was also analyzed to reveal sub-region relevance to each prediction class. Image occlusion with blank sub-regions [24,51,79] and healthy-looking sub-regions [95] was used to find the most informative and relevant sub-regions for classification and detection tasks.…”

Section: The Use Of An Attention Mechanism Was the Most Commonmentioning

confidence: 99%

“…Image visualizationbased techniques to achieve transparency were most commonly considered in image classification problems. Common ways of retrieving relevance information were: Visual relevancy through attention [1, 3, 30, 34, 5, 38-40, 68, 83, 89, 91, 106]; region occlusion by blank areas [24,79] or healthy-looking regions [95]; and other techniques such as supervision of activated image regions by clinically relevant areas [54,87,43,75,99,102,111], and image similarity [31]. Feature-based computer vision transparency priors focused on the impact of feature evolution or perturbation on the decoded output.…”

Section: Pr: Priorsmentioning

confidence: 99%

“…The complex nature of both 3D imaging in radiology and pathological images makes image analysis tasks more time consuming than 2D image analysis that is more prevalent in other specialities, such as dermatology, which motivates transparency as an alternative to complete human image analysis to save time while retaining trustworthiness. In detail, classification problems in 3D radiological images and pathological images included abnormality detection in computed tomography (CT) scans [3,5,61,47,89,107,111,112], MRIs [34,11,38,51,59,87,85,50,95,77,78,98,100,104], pathology images [1,24,26,27,30,34,37,40,82,84,50,74,76,108,5] and positron emission tomography (PET) images [68]. Mammography dominated the 2D radiology image applications [60,88,44,45,86,96,99,…”

Section: T: Taskmentioning

confidence: 99%

See 2 more Smart Citations

INTRPRT: A Systematic Review of and Guidelines for Designing and Validating Transparent AI in Medical Image Analysis

Chen¹,

Gómez²,

Huang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Transparency in Machine Learning (ML), including interpretable or explainable ML, attempts to reveal the working mechanisms of complex models, including deep neural networks. Transparent ML promises to advance human factors engineering goals of human-centered AI, such as increasing trust or reducing automation bias, in the target users. However, a core requirement in achieving these aims is that the method is indeed transparent to the users. From a human-centered design perspective, transparency is not a property of the ML model but an affordance, i. e., a relationship between algorithm and user; as a result, iterative prototyping and evaluation with users is critical to attaining adequate solutions that afford transparency. However, following human-centered design principles in highly specialized and high stakes domains, such as healthcare and medical image analysis, is challenging due to the limited availability of and access to end users, such as domain experts, providers, or patients. This dilemma is further exacerbated by the high knowledge imbalance between ML designers and those end users, which implies an even greater need for iterated development.To investigate the state of transparent ML in medical image analysis, we conducted and now present a systematic review of the literature. Our review reveals multiple severe shortcomings in the design and validation of transparent ML for medical image analysis applications. We find that most studies to date approach transparency as a property of the model itself, similar to task performance, without considering end users during neither development nor evaluation. Despite the considerable difference between the roles and knowledge of ML developers and clinical stakeholders, no study reported formative user research to inform the design and development of transparent ML models; moreover, only a few studies validated transparency claims through empirical user evaluations. The oversight of considering ML transparency as a relationship with end users, the lack of user research, and the sporadic validation of transparency claims put contemporary research on transparent ML for medical image analysis at risk of being incomprehensible to users, and thus, clinically irrelevant. To alleviate these shortcomings in forthcoming research while acknowledging the challenges of human-centered design in healthcare, we introduce the INTRPRT guideline, a systematic design directive for transparent ML systems in medical image analysis. To bridge the disconnect between ML system designers and end users and avoid misspending costly development efforts on models that are finally validated as non-transparent to end users, the INTRPRT guideline suggests formative user research as the first step of transparent model design to understand user needs and domain requirements. Following this process produces evidence to support * Joint first authors.Preprint. Under review.

show abstract

Pseudo-class part prototype networks for interpretable breast cancer classification

Choukali,

Amirani,

Valizadeh

et al. 2024

Sci Rep

View full text Add to dashboard Cite

Interpretability in machine learning has become increasingly important as machine learning is being used in more and more applications, including those with high-stakes consequences such as healthcare where Interpretability has been regarded as a key to the successful adoption of machine learning models. However, using confounding/irrelevant information in making predictions by deep learning models, even the interpretable ones, poses critical challenges to their clinical acceptance. That has recently drawn researchers’ attention to issues beyond the mere interpretation of deep learning models. In this paper, we first investigate application of an inherently interpretable prototype-based architecture, known as ProtoPNet, for breast cancer classification in digital pathology and highlight its shortcomings in this application. Then, we propose a new method that uses more medically relevant information and makes more accurate and interpretable predictions. Our method leverages the clustering concept and implicitly increases the number of classes in the training dataset. The proposed method learns more relevant prototypes without any pixel-level annotated data. To have a more holistic assessment, in addition to classification accuracy, we define a new metric for assessing the degree of interpretability based on the comments of a group of skilled pathologists. Experimental results on the BreakHis dataset show that the proposed method effectively improves the classification accuracy and interpretability by respectively $$8 \%$$ 8 % and $$18 \%$$ 18 % . Therefore, the proposed method can be seen as a step toward implementing interpretable deep learning models for the detection of breast cancer using histopathology images.

show abstract

Approximate Explanations for Classification of Histopathology Patches

Cited by 3 publications

References 9 publications

Explainable medical imaging AI needs human-centered design: guidelines and evidence from a systematic review

Explainable medical imaging AI needs human-centered design: guidelines and evidence from a systematic review

INTRPRT: A Systematic Review of and Guidelines for Designing and Validating Transparent AI in Medical Image Analysis

Pseudo-class part prototype networks for interpretable breast cancer classification

Contact Info

Product

Resources

About