Accuracy is in the eyes of the pathologist: The visual interpretive process and diagnostic accuracy with digital whole slide images

Abstract

Digital whole slide imaging is an increasingly common medium in pathology, with application to education, telemedicine, and rendering second opinions. It has also made it possible to use eye tracking devices to explore the dynamic visual inspection and interpretation of histopathological features of tissue while pathologists review cases. Using whole slide images, the present study examined how a pathologist’s diagnosis is influenced by fixed case-level factors, their prior clinical experience, and their patterns of visual inspection. Participating pathologists interpreted one of two test sets, each containing 12 digital whole slide images of breast biopsy specimens. Cases represented four diagnostic categories as determined via expert consensus: benign without atypia, atypia, ductal carcinoma in situ (DCIS), and invasive cancer. Each case included one or more regions of interest (ROIs) previously determined as of critical diagnostic importance. During pathologist interpretation we tracked eye movements, viewer tool behavior (zooming, panning), and interpretation time. Models were built using logistic and linear regression with generalized estimating equations, testing whether variables at the level of the pathologists, cases, and visual interpretive behavior would independently and/or interactively predict diagnostic accuracy and efficiency. Diagnostic accuracy varied as a function of case consensus diagnosis, replicating earlier research. As would be expected, benign cases tended to elicit false positives, and atypia, DCIS, and invasive cases tended to elicit false negatives. Pathologist experience levels, case consensus diagnosis, case difficulty, eye fixation durations, and the extent to which pathologists’ eyes fixated within versus outside of diagnostic ROIs, all independently or interactively predicted diagnostic accuracy. Higher zooming behavior predicted a tendency to over-interpret benign and atypia cases, but not DCIS cases. Efficiency was not predicted by pathologist- or visual search-level variables. Results provide new insights into the medical interpretive process and demonstrate the complex interactions between pathologists and cases that guide diagnostic decision-making. Implications for training, clinical practice, and computer-aided decision aids are considered.

Eye-tracking for assessing medical image interpretation: A pilot feasibility study comparing novice vs expert cardiologists

“…Thus, novices display longer and more in-depth viewing relative to experts, though when diagnostic decisions were ultimately made, they did not show higher accuracy. This complements and extends similar results found with radiologists and pathologists viewing static images [ 2 , 4 , 16 , 32 , 44 , 45 ]. Using a video summarization technique derived from multimedia informatics, there was evidence that novices not only fixate more on critical frames than experts, but they also consistently (with one exception, RAO30) fixate on each frame’s critical region.…”

“…To increase the efficiency of interpreting eye movements over dynamic scenes, researchers can temporally couple eye tracking with logged interface behaviour to relate eye position to specific video frames [ 32 – 34 ]. To prioritize video frames, a behaviour analysis can assist in inferring a viewer’s interest in certain video frames: using the ShotRank technique, the frequency and duration of viewing video frames indicates interest in the information available on those frames [ 35 – 37 ].…”

An Expert Derived Feedforward Histology Module Improves Pattern Recognition Efficiency in Novice Students

“…Although not significant, the experimental group participants also had on average, a lower number of fixations and a shorter total scan path distance. This pattern of fixation and saccade parameters are consistent with a more efficient search pattern and higher visual literacy (Henderson, ; Krupinski et al, ; Krupinski et al, ; Brunyé et al, ; Jaarsma et al, ; Brunyé et al, ).…”

“…Thus, when performing visual tasks, multiple eye movements to different locations typically occur to provide high‐resolution information from different parts of the visual scene. Studies over the past decade have established profiles of both eye movement parameters and performance parameters for novices and experts in various visual subjects, such as pathology and radiology, which require visual search of an image to reach a diagnosis or conclusion (Krupinski et al, ; Roa‐Pena et al, ; Krupinski et al, ; Brunyé et al, ; Jaarsma et al, ; Zumwalt et al, ; Brunyé et al, ; Hanley et al, ; Borg et al, ; Warren et al, ). Although it can vary depending on the task (identification, interpretation, or both), these studies have generally shown that when examining a digital image, experts utilize fewer fixations and saccades, have longer fixation durations on task‐relevant areas, and complete identification tasks with greater accuracy and in less time when compared to novices.…”

Report on computational assessment of Tumor Infiltrating Lymphocytes from the International Immuno-Oncology Biomarker Working Group

“…TIL-WG is an international coalition of pathologists, oncologists, statisticians, and data scientists that standardize the assessment of Immuno-Oncology Biomarkers to aid pathologists, clinicians, and researchers in their research and daily practice. The value of these guidelines was highlighted in two studies systematically examining VTA reproducibility 7,13 . Nevertheless, VTA continues to have inherent limitations that cannot be fully addressed through standardization and training, including: 1. visual assessment will always have some degree of inter-reader variability; 2. the time constraints of routine practice make comprehensive assessment of large tissue sections challenging 7,13 ; 3. perceptual limitations may introduce bias in VTA, for example, the same TILs density is perceived to be higher if there is limited stroma.…”