Experimental system for measurement of radiologists’ performance by visual search task

Abstract

PurposeDetective performance of radiologists for “obvious” targets should be evaluated by visual search task instead of ROC analysis, but visual task have not been applied to radiology studies. The aim of this study was to set up an environment that allows visual search task in radiology, to evaluate its feasibility, and to preliminarily investigate the effect of career on the performance.Materials and methodsIn a darkroom, ten radiologists were asked to answer the type of lesion by pressing buttons, when images without lesions, with bulla, ground-glass nodule, and solid nodule were randomly presented on a display. Differences in accuracy and reaction times depending on board certification were investigated.ResultsThe visual search task was successfully and feasibly performed. Radiologists were found to have high sensitivity, specificity, positive predictive values and negative predictive values in non-board and board groups. Reaction time was under 1 second for all target types in both groups. Board radiologists were significantly faster in answering for bulla, but there were no significant differences for other targets and values.ConclusionWe developed an experimental system that allows visual search experiment in radiology. Reaction time for detection of bulla was shortened with experience.

Evaluation of the diagnostic accuracy of four-view radiography and conventional computed tomography analysing sacral and pelvic fractures in dogs

“…Factors noted influencing the performance included the modality, reconstruction methods, and years of experience. It was suggested that a higher level of experience and training result in a higher level of interreader reliability and higher sensitivity in lesion detection (14)(15)(16).…”

Visual Search Performance and Strategy for Three-Dimensional Visualization Systems: Impact of Radiologist Experience

“…Previously, the domain-general temporal aspects of radiologists' performance were examined using eye-tracking [33,35], and by recording the reaction time [29,30] and trial completion time [34]. However, no differences were reported for gaze fixation characteristics [35], and radiologists demonstrated considerably faster reactions than other participants when detecting slight changes in non-radiological images [29] and finding low contrast targets [28,30], which allowed us to assume that the physical properties of stimuli might play a critical role in the temporal evaluation of performance depending on radiologist experience. In our study, we saw a tendency for students to consume more time compared to radiologists and residents when target-distractor similarity was high, however, no significance was reached.…”

“…However, the investigation of perceptual processes for edited medical images [35,36] and non-medical search items [28,32,[37][38][39][40] clarified that the superior skills of radiologists were shown only in the presence of specific physical properties of visual stimulus. Namely, considerable differences were revealed in the temporal aspects of task outcome when low contrast stimuli were employed [28,35], suggesting that contrast sensitivity might be enhanced in radiologists. Therefore, one can expect differences in the performance of radiologists when not only viewing domain-specific images, but also domain-general images.…”

Temporal Characteristics of Radiologists' and Novices' Lesion Detection in Viewing Medical Images Presented Rapidly and Sequentially

“…The suspicious areas include obvious locations as well as subtle or hidden locations such as lesions masked and/or overlapped by other anatomical features. Although many studies in radiology have discussed the detection of uncertain (i.e., not obvious) lesions (e.g., Evans et al, 2011b , 2013 ; see also Metz, 1978 ), visual search for obvious lesions is also an important issue (e.g., Carmody et al, 1981 ; Oestmann et al, 1988 ; Maeda et al, 2013 ; see also Kundel, 2006 ). Actually, missing the lesions in a search often occurs even when the lesions are relatively obvious (e.g., scanning errors : observers do not fixate on the lesions at all; see Kundel et al, 1978 ).…”

“…In this process, a radiologist typically scrolls quickly through a stack of 2-D CT images, which are thin slices of the 3-D volume of an organ. This is known as “stack viewing.” Although a number of studies have examined search strategies used in viewing single 2-D medical images (e.g., Carmody et al, 1981 ; Kundel et al, 1991 ; Ellis et al, 2006 ; Manning et al, 2006 ; Evans et al, 2013 ; Maeda et al, 2013 ; Nakashima et al, 2013 , 2015 ), little is known about searches of 3-D medical images. Radiologists, who engage in stack viewing to detect lesions in an organ, usually scroll through the series of 2-D images very quickly at a more or less constant speed.…”