Eye Gaze Pattern Analysis of Whole Slide Image Viewing Behavior in PathEdEx Platform

Abstract

Visual interpretation of histopathological slides is a complex process that may lead to diagnostic pitfalls. We have started a study to quantify and analyze pathologist's eye gaze patterns while viewing whole slide images (WSI) of specimens on computer monitor in order to explore and uncover diagnostic heuristics. As part of our ongoing research, we are developing a digital pathology integrative platform (DPIP) to provide a training and research resource that includes an online case-based training site, a collection of digitized whole slide images, in-house developed microscopy image analysis software for full-and semi-automated annotation of H&E and IHC images, and eye gaze tracking hardware to collect viewing behavior data. Our online, interactive hematopathology training site has WSI viewing capability, and a realistic case-based diagnosis workflow for training of pathologists, and it also enables collection of data for research in diagnostics. Our training site, named PathEdEx (pathedex.com), consists of 43 deidentified real patient cases from University of Missouri Ellis Fischel Cancer Center, and provides an interactive diagnostic workflow that allows users to navigate and review the available patient data including all H&E and IHC whole slides. Figure 1 shows the components of the platform. Each case contains patient history, all laboratory tests that are available for the patient, and high resolution images of H&E and several IHC stains scanned by a whole slide scanner that produces GB size images. The user is allowed to navigate through the case data to experience a realistic diagnostic workflow, and the navigation behavior as well as final diagnosis of each user is recorded and scored.We collected eye gaze tracking data of pathologists and residents in order to quantify and analyze viewing patterns to relate the differences in their viewing behavior to their levels of experience. Nine users with different levels of experience (medical students, first, second, fourth year residents, and pathologists) have navigated through four cases in order to make a diagnosis. The eye gaze tracking data of the participants were collected by using a Tobii EyeX tracker device while they were interacting with the whole slide images using the integrated WSI viewer by panning and zooming in a virtual microscope manner. Figure 2 shows a sample of eye gaze data superposed on the low magnification whole slide images. Eye gaze and navigational data were collected at a rate of about 30 samples/s and analyzed at different zoom levels in the corresponding whole slides to extract the fixation points and a variety of features to explore the best features related to expertise levels. We have found that three features are significant in terms of representing viewing behavior of participants on a case by case basis.Analyzing gaze tracking data is becoming more available in radiology and pathology [1,2,3,4]. Many studies analyze the captured gaze data with respect to the regions of interest (ROI) marked in the medical images; gaze tim...

On Smart Gaze Based Annotation of Histopathology Images for Training of Deep Convolutional Neural Networks

“…Both studies concluded that the gaze of expert/fully-trained pathologists converged much more quickly towards diagnostically relevant ROIs than junior or trainee pathologists. PathEdEx, an online tool that records and automatically groups the gaze patterns of pathologists was introduced in [17]. The resulting clusters, along with clinical notes added by expert pathologists, were then employed for training of pathology students and trainees.…”