Mouse cursor movement and eye tracking data as an indicator of pathologists' attention when viewing digital whole slide images

Abstract

Context:Digital pathology has the potential to dramatically alter the way pathologists work, yet little is known about pathologists’ viewing behavior while interpreting digital whole slide images. While tracking pathologist eye movements when viewing digital slides may be the most direct method of capturing pathologists’ viewing strategies, this technique is cumbersome and technically challenging to use in remote settings. Tracking pathologist mouse cursor movements may serve as a practical method of studying digital slide interpretation, and mouse cursor data may illuminate pathologists’ viewing strategies and time expenditures in their interpretive workflow.Aims:To evaluate the utility of mouse cursor movement data, in addition to eye-tracking data, in studying pathologists’ attention and viewing behavior.Settings and Design:Pathologists (N = 7) viewed 10 digital whole slide images of breast tissue that were selected using a random stratified sampling technique to include a range of breast pathology diagnoses (benign/atypia, carcinoma in situ, and invasive breast cancer). A panel of three expert breast pathologists established a consensus diagnosis for each case using a modified Delphi approach.Materials and Methods:Participants’ foveal vision was tracked using SensoMotoric Instruments RED 60 Hz eye-tracking system. Mouse cursor movement was tracked using a custom MATLAB script.Statistical Analysis Used:Data on eye-gaze and mouse cursor position were gathered at fixed intervals and analyzed using distance comparisons and regression analyses by slide diagnosis and pathologist expertise. Pathologists’ accuracy (defined as percent agreement with the expert consensus diagnoses) and efficiency (accuracy and speed) were also analyzed.Results:Mean viewing time per slide was 75.2 seconds (SD = 38.42). Accuracy (percent agreement with expert consensus) by diagnosis type was: 83% (benign/atypia); 48% (carcinoma in situ); and 93% (invasive). Spatial coupling was close between eye-gaze and mouse cursor positions (highest frequency ∆x was 4.00px (SD = 16.10), and ∆y was 37.50px (SD = 28.08)). Mouse cursor position moderately predicted eye gaze patterns (Rx = 0.33 and Ry = 0.21).Conclusions:Data detailing mouse cursor movements may be a useful addition to future studies of pathologists’ accuracy and efficiency when using digital pathology.

Eye Movements as an Index of Pathologist Visual Expertise: A Pilot Study

“…Participants were divided into three groups [25]–[27]: 1) Novices were comprised of current residents with limited breast pathology experience (n = 3); 2) Intermediates were comprised of faculty members specializing in dermatopathology and general anatomic pathology (n = 2), and 3) Experts were comprised of faculty who specialized in breast pathology (n = 2).…”

Machine Learning Methods for Histopathological Image Analysis

“…Web server [42,43] Tracking pathologists' behavior Eye tracking [44], mouse tracking [45] and viewport tracking [46] Active learning Uncertainly sampling [43], Query-by-Committee [47], variance reduction [48] and hypothesis space reduction [49] Multiple instance learning Boosting-based [50,51], deep weak supervision [52] and structured support vector machines (SVM) [53] Semi-supervised learning Manifold learning [30] and SVM [54] Transfer learning Feature extraction [55], fine-tuning [16,56,57]…”

“…Another interesting idea to reduce working time is to automatically localize ROIs during diagnosis, which uses the usual working time for diagnosis as labeling by tracking pathologists' behavior. This approach tracks pathologists' eye movement [44], mouse cursor positions [45] and change in viewport [46]. However, localizing ROIs accurately from these tracking data is not always easy since pathologist's do not always spend time looking at ROIs, and boundary information obtained by these approaches tends to be less clear.…”

Students' performance during practical examination on whole slide images using view path tracking

“…The software-based tracking method for WSIs appeared to be a good choice for the exam session environment. This approach has certain advantages over solutions described in [ 9 , 11 ], which involve eye movement tracking. Once implemented on a WSI software platform, our method is easy to adapt and requires no changes in the way how WSIs are normally viewed.…”

“…Using a software method is an alternative. It has already been suggested that signals collected from regular WSI viewing hardware, like mouse cursor position [ 11 ] and data from slide navigation [ 12 ], may have potential in identification of viewing behavior.…”