Localization of Diagnostically Relevant Regions of Interest in Whole Slide Images: a Comparative Study

Mercan, Ezgi; Aksoy, Selim; Shapiro, Linda G.; Weaver, Donald L.; Brunyé, Tad T.; Elmore, Joann G.

doi:10.1007/s10278-016-9873-1

Cited by 59 publications

(53 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As eye tracking systems become increasingly prevalent and flexible, and less obtrusive and expensive, they become more feasible for incorporation into classrooms and clinics 52,55 . A continuing challenge to this goal, however, is developing more robust, flexible, reliable, and clinically relevant algorithms for automated image processing and eye movement interpretation 56 .…”

Section: Discussionmentioning

confidence: 99%

Accuracy is in the eyes of the pathologist: The visual interpretive process and diagnostic accuracy with digital whole slide images

Bruny

Mercan

Weaver

et al. 2017

Journal of Biomedical Informatics

Self Cite

View full text Add to dashboard Cite

Digital whole slide imaging is an increasingly common medium in pathology, with application to education, telemedicine, and rendering second opinions. It has also made it possible to use eye tracking devices to explore the dynamic visual inspection and interpretation of histopathological features of tissue while pathologists review cases. Using whole slide images, the present study examined how a pathologist’s diagnosis is influenced by fixed case-level factors, their prior clinical experience, and their patterns of visual inspection. Participating pathologists interpreted one of two test sets, each containing 12 digital whole slide images of breast biopsy specimens. Cases represented four diagnostic categories as determined via expert consensus: benign without atypia, atypia, ductal carcinoma in situ (DCIS), and invasive cancer. Each case included one or more regions of interest (ROIs) previously determined as of critical diagnostic importance. During pathologist interpretation we tracked eye movements, viewer tool behavior (zooming, panning), and interpretation time. Models were built using logistic and linear regression with generalized estimating equations, testing whether variables at the level of the pathologists, cases, and visual interpretive behavior would independently and/or interactively predict diagnostic accuracy and efficiency. Diagnostic accuracy varied as a function of case consensus diagnosis, replicating earlier research. As would be expected, benign cases tended to elicit false positives, and atypia, DCIS, and invasive cases tended to elicit false negatives. Pathologist experience levels, case consensus diagnosis, case difficulty, eye fixation durations, and the extent to which pathologists’ eyes fixated within versus outside of diagnostic ROIs, all independently or interactively predicted diagnostic accuracy. Higher zooming behavior predicted a tendency to over-interpret benign and atypia cases, but not DCIS cases. Efficiency was not predicted by pathologist- or visual search-level variables. Results provide new insights into the medical interpretive process and demonstrate the complex interactions between pathologists and cases that guide diagnostic decision-making. Implications for training, clinical practice, and computer-aided decision aids are considered.

show abstract

Section: Discussionmentioning

confidence: 99%

Accuracy is in the eyes of the pathologist: The visual interpretive process and diagnostic accuracy with digital whole slide images

Bruny

Mercan

Weaver

et al. 2017

Journal of Biomedical Informatics

Self Cite

View full text Add to dashboard Cite

show abstract

“…Of particular interest in WSI analysis is the use of multiscale patch representations that allow concurrent use of high-zoom patches that capture cellular level information with lower-zoom patches that capture global interdependence of tissue structures [16][17][18]. Bejnordi et al used multiscale patch representation of WSIs to build highly accurate context-aware stacked convolutional neural networks (CNN) for distinguishing between invasive ductal carcinomas (IDC) and benign ductal carcinoma in situ (DCIS) [19].…”

Section: Introductionmentioning

confidence: 99%

A deep learning image-based intrinsic molecular subtype classifier of breast tumors reveals tumor heterogeneity that may affect survival

Jaber¹,

Song

Taylor

et al. 2020

Breast Cancer Res

View full text Add to dashboard Cite

Background: Breast cancer intrinsic molecular subtype (IMS) as classified by the expression-based PAM50 assay is considered a strong prognostic feature, even when controlled for by standard clinicopathological features such as age, grade, and nodal status, yet the molecular testing required to elucidate these subtypes is not routinely performed. Furthermore, when such bulk assays as RNA sequencing are performed, intratumoral heterogeneity that may affect prognosis and therapeutic decision-making can be missed. Methods: As a more facile and readily available method for determining IMS in breast cancer, we developed a deep learning approach for approximating PAM50 intrinsic subtyping using only whole-slide images of H&E-stained breast biopsy tissue sections. This algorithm was trained on images from 443 tumors that had previously undergone PAM50 subtyping to classify small patches of the images into four major molecular subtypes-Basal-like, HER2-enriched, Luminal A, and Luminal B-as well as Basal vs. non-Basal. The algorithm was subsequently used for subtype classification of a held-out set of 222 tumors. Results: This deep learning image-based classifier correctly subtyped the majority of samples in the held-out set of tumors. However, in many cases, significant heterogeneity was observed in assigned subtypes across patches from within a single whole-slide image. We performed further analysis of heterogeneity, focusing on contrasting Luminal A and Basal-like subtypes because classifications from our deep learning algorithm-similar to PAM50-are associated with significant differences in survival between these two subtypes. Patients with tumors classified as heterogeneous were found to have survival intermediate between Luminal A and Basal patients, as well as more varied levels of hormone receptor expression patterns. Conclusions: Here, we present a method for minimizing manual work required to identify cancer-rich patches among all multiscale patches in H&E-stained WSIs that can be generalized to any indication. These results suggest that advanced deep machine learning methods that use only routinely collected whole-slide images can approximate RNAseq-based molecular tests such as PAM50 and, importantly, may increase detection of heterogeneous tumors that may require more detailed subtype analysis.

show abstract

“…While investigators are just beginning to study pathologists’ WSI viewing behavior and draw conclusions from a visualized microscopic examination,[29] we were able to notice three general viewing patterns in our trainees. Some residents reviewed the tissue broadly at low power, then zoomed in once on a region of interest (B, D, E).…”

Section: Discussionmentioning

confidence: 87%

RecutClub.com: An Open Source, Whole Slide Image-based Pathology Education System

Christensen¹,

Lee²,

Thrall³

et al. 2017

Journal of Pathology Informatics

View full text Add to dashboard Cite

Background:Our institution's pathology unknown conferences provide educational cases for our residents. However, the cases have not been previously available digitally, have not been collated for postconference review, and were not accessible to a wider audience. Our objective was to create an inexpensive whole slide image (WSI) education suite to address these limitations and improve the education of pathology trainees.Materials and Methods:We surveyed residents regarding their preference between four unique WSI systems. We then scanned weekly unknown conference cases and study set cases and uploaded them to our custom built WSI viewer located at RecutClub.com. We measured site utilization and conference participation.Results:Residents preferred our OpenLayers WSI implementation to Ventana Virtuoso, Google Maps API, and OpenSlide. Over 16 months, we uploaded 1366 cases from 77 conferences and ten study sets, occupying 793.5 GB of cloud storage. Based on resident evaluations, the interface was easy to use and demonstrated minimal latency. Residents are able to review cases from home and from their mobile devices. Worldwide, 955 unique IP addresses from 52 countries have viewed cases in our site.Conclusions:We implemented a low-cost, publicly available repository of WSI slides for resident education. Our trainees are very satisfied with the freedom to preview either the glass slides or WSI and review the WSI postconference. Both local users and worldwide users actively and repeatedly view cases in our study set.

show abstract

Localization of Diagnostically Relevant Regions of Interest in Whole Slide Images: a Comparative Study

Cited by 59 publications

References 48 publications

Accuracy is in the eyes of the pathologist: The visual interpretive process and diagnostic accuracy with digital whole slide images

Accuracy is in the eyes of the pathologist: The visual interpretive process and diagnostic accuracy with digital whole slide images

A deep learning image-based intrinsic molecular subtype classifier of breast tumors reveals tumor heterogeneity that may affect survival

RecutClub.com: An Open Source, Whole Slide Image-based Pathology Education System

Contact Info

Product

Resources

About