Foveated Model Observers for Visual Search in 3D Medical Images

Abstract

Model observers have a long history of success in predicting human observer performance in clinically-relevant detection tasks. New 3D image modalities provide more signal information but vastly increase the search space to be scrutinized. Here, we compared standard linear model observers (ideal observers, non-pre-whitening matched filter with eye filter, and various versions of Channelized Hotelling models) to human performance searching in 3D 1/f 2.8 filtered noise images and assessed its relationship to the more traditional location known exactly detection tasks and 2D search. We investigated two different signal types that vary in their detectability away from the point of fixation (visual periphery). We show that the influence of 3D search on human performance interacts with the signal's detectability in the visual periphery. Detection performance for signals difficult to detect in the visual periphery deteriorates greatly in 3D search but not in 3D location known exactly and 2D search. Standard model observers do not predict the interaction between 3D search and signal type. A proposed extension of the Channelized Hotelling model (foveated search model) that processes the image with reduced spatial detail away from the point of fixation, explores the image through eye movements, and scrolls across slices can successfully predict the interaction observed in humans and also the types of errors in 3D search. Together, the findings highlight the need for foveated model observers for image quality evaluation with 3D search.

More scanning, but not zooming, is associated with diagnostic accuracy in evaluating digital breast pathology slides

“…The basic science literature suggests that this apparent discrepancy is likely due to important differences in the nature of the images being evaluated by clinicians. Although there has been recent modeling work to adapt ideal observer models from 2D medical images to the volumetric stack images like chest CT ( Lago, Abbey, & Eckstein, 2021 ), we are not aware of work to adapt this work to zoomable high-resolution images that are found in digital pathology.…”

A Robust 3-D Medical Watermarking Based on Wavelet Transform for Data Protection

“…Calculate the corresponding eigenvector e 1 , e 2 , e 3 based on the formula 3), where C v is a symmetric matrix, and the eigenvectors of the matrix are orthogonal to each other. The spatial coordinate system, namely PCA coordinate system, can be constructed according to the eigenvectors.…”

“…With the huge increase of 5G technology in communication speed, more and more medical images are being presented to doctors and patients in the form of 3D models. By performing operations such as scaling, rotating, and moving on these models [3,4], doctors can more intuitively find out whether there are lesions in tissues or organs, and further conduct comprehensive pathological inquiry, technical communication, demonstration and surgery simulation [5]. In order to ensure the safe transmission and storage of these medical images, watermark technology [6][7][8][9] is applied to medical data to solve the possible malicious tampering or attack in the process of medical data transmission.…”

Medical image quality metrics for foveated model observers

“…Both panels include human performance as measured by the empirical 𝑑′ calculated from seven observers. The panels also show the predicted 𝑑′ using the full implementation of a Foveated Search Model (FSM) that executes the same eye movements and scrolls of each human observer for the corresponding trial [62].…”