Global processing provides malignancy evidence complementary to the information captured by humans or machines following detailed mammogram inspection

Abstract

The information captured by the gist signal, which refers to radiologists’ first impression arising from an initial global image processing, is poorly understood. We examined whether the gist signal can provide complementary information to data captured by radiologists (experiment 1), or computer algorithms (experiment 2) based on detailed mammogram inspection. In the first experiment, 19 radiologists assessed a case set twice, once based on a half-second image presentation (i.e., gist signal) and once in the usual viewing condition. Their performances in two viewing conditions were compared using repeated measure correlation (rm-corr). The cancer cases (19 cases × 19 readers) exhibited non-significant trend with rm-corr = 0.012 (p = 0.82, CI: −0.09, 0.12). For normal cases (41 cases × 19 readers), a weak correlation of rm-corr = 0.238 (p < 0.001, CI: 0.17, 0.30) was found. In the second experiment, we combined the abnormality score from a state-of-the-art deep learning-based tool (DL) with the radiological gist signal using a support vector machine (SVM). To obtain the gist signal, 53 radiologists assessed images based on half-second image presentation. The SVM performance for each radiologist and an average reader, whose gist responses were the mean abnormality scores given by all 53 readers to each image was assessed using leave-one-out cross-validation. For the average reader, the AUC for gist, DL, and the SVM, were 0.76 (CI: 0.62–0.86), 0.79 (CI: 0.63–0.89), and 0.88 (CI: 0.79–0.94). For all readers with a gist AUC significantly better than chance-level, the SVM outperformed DL. The gist signal provided malignancy evidence with no or weak associations with the information captured by humans in normal radiologic reporting, which involves detailed mammogram inspection. Adding gist signal to a state-of-the-art deep learning-based tool improved its performance for the breast cancer detection.

Using global feedback to induce learning of gist of abnormality in mammograms

“…Together, these findings solidly emphasize the need for continued research into medical perceptual expertise with human observers in its own right, especially into more ambiguous global signals that would be vital for early cancer detection. But it also reinforces the need of combining these lines of research with the thriving field of machine learning research, especially since recent research has suggested benefits of combining radiologists’ gist ratings with machine learning models to reach higher levels of performance than either could on their own (Gandomkar et al, 2021 ; Wurster et al, 2019 ).…”

Spotting lesions in thorax X-rays at a glance: holistic processing in radiology

“…3 , left panel). After all, it is known that the global gist is independent of the presentation time (Gandomkar et al, 2021 ; Raat et al, 2021 ). It is, however, difficult to exclude the possibility that both processes interact with each other to enable the performance, as we do not know whether the radiologists could indicate where exactly the lesions were present.…”

“…The research on gist abnormality has repeatedly shown that the unlocalized sense for the presence of abnormalities can be extracted in situations where there the signal is degraded or not even clearly visible, such as from healthy halves of the stimuli (Evans et al, 2016) or stimuli that are currently normal but will develop abnormalities in future (Evans et al, 2019). Similarly, the initial sense of abnormalities does not seem to improve with time, that is, it stays at the same level as it was with a flash presentation (Gandomkar et al, 2021;Raat et al, 2021). This is clearly a speculative assumption, but, to our knowledge, there are no studies on the global gist which featured the inversion effect, nor in which the inversion condition was combined with differing presentation time.…”

“…Unlike the postulated global impression during the holistic processing, which is supposed to improve with additional exposure to the stimuli (Kundel & Nodine, 1975 ; Oestmann et al, 1993 ), the global gist for abnormalities seems to be static. Additional time, including unlimited viewing time, does not improve on the initial hunch about the presence of abnormalities in medical images (Gandomkar et al, 2021 ; Raat et al, 2021 ).…”

Multiple expressions of “expert” abnormality gist in novices following perceptual learning

“…Recently, this view of the nature of the global signal has been challenged (Evans et al, 2013 , 2016 , 2019 ). Rather than containing location information, Evans et al ( 2013 , 2016 , 2019 ) and Gandomkar et al ( 2021 ) have argued that the first process is a global abnormality signal devoid of location information. This first global signal (“gist”) simply provides information that something is abnormal based on a global implicit extraction of statistics across the whole image allowing for abnormality detection without containing any location information about where the abnormality lies or even why the image is being called abnormal.…”