Influence of study design on digital pathology image quality evaluation: the need to define a clinical task

Abstract

Abstract. Despite the current rapid advance in technologies for whole slide imaging, there is still no scientific consensus on the recommended methodology for image quality assessment of digital pathology slides. For medical images in general, it has been recommended to assess image quality in terms of doctors' success rates in performing a specific clinical task while using the images (clinical image quality, cIQ). However, digital pathology is a new modality, and already identifying the appropriate task is difficult. In an alternative common approach, humans are asked to do a simpler task such as rating overall image quality (perceived image quality, pIQ), but that involves the risk of nonclinically relevant findings due to an unknown relationship between the pIQ and cIQ. In this study, we explored three different experimental protocols: (1) conducting a clinical task (detecting inclusion bodies), (2) rating image similarity and preference, and (3) rating the overall image quality. Additionally, within protocol 1, overall quality ratings were also collected (task-aware pIQ). The experiments were done by diagnostic veterinary pathologists in the context of evaluating the quality of hematoxylin and eosin-stained digital pathology slides of animal tissue samples under several common image alterations: additive noise, blurring, change in gamma, change in color saturation, and JPG compression. While the size of our experiments was small and prevents drawing strong conclusions, the results suggest the need to define a clinical task. Importantly, the pIQ data collected under protocols 2 and 3 did not always rank the image alterations the same as their cIQ from protocol 1, warning against using conventional pIQ to predict cIQ. At the same time, there was a correlation between the cIQ and task-aware pIQ ratings from protocol 1, suggesting that the clinical experiment context (set by specifying the clinical task) may affect human visual attention and bring focus to their criteria of image quality. Further research is needed to assess whether and for which purposes (e.g., preclinical testing) task-aware pIQ ratings could substitute cIQ for a given clinical task. © The Authors. Published by SPIE under aCreative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

Impact of JPEG 2000 compression on deep convolutional neural networks for metastatic cancer detection in histopathological images

“…9 For studying the impact of lossy compression on the diagnostic performance of human experts, several studies have been reported. 8,[10][11][12][13][14][15][16] Mostly, they reported that the human visual perception is to some extent robust against image quality degradation. However, there is not a generally accepted tolerance level with respect to the diagnostic accuracy.…”

Stable and discriminating features are predictive of cancer presence and Gleason grade in radical prostatectomy specimens: a multi-site study

“…While these approaches have the potential to help identify the appropriate feature selection approach, they do not by themselves provide any real insight on the stability of the actual features across multiple different sites. In the digital pathology space, there have been recent studies documenting sources of pre-analytic variation that influence the color of the resulting tissue scanned images 16 – 18 , but little by way of evaluation measures for consistency and reproducibility of tissue derived image features.…”

Comparative study of the methodologies used for subjective medical image quality assessment

“…Finally, Platiša et al explored the impact of distinct experimental protocols on the image quality evaluation of digital pathology slides [107]. Their dataset was comprised of three images of animal pathology samples, i.e., two of gastric fundic glands and one of liver.…”