Guidelines for Quantitative Evaluation of Medical Visualizations on the Example of 3D Aneurysm Surface Comparisons

Abstract

Medical visualizations are highly adapted to a specific medical application scenario. Therefore, many researchers conduct qualitative evaluations with a low number of physicians or medical experts to assess the benefits of their visualization technique. Although this type of research has advantages, it is difficult to reproduce and can be subjectively biased. This makes it problematic to quantify the benefits of a new visualization technique. Quantitative evaluation can objectify research and help bringing new visualization techniques into clinical practice. To support researchers, we present guidelines to quantitatively evaluate medical visualizations, considering specific characteristics and difficulties. We demonstrate the adaptation of these guidelines on the example of comparative aneurysm surface visualizations. We developed three visualization techniques to compare aneurysm volumes. The visualization techniques depict two similar, but not identical aneurysm surface meshes. In a user study with 34 participants and five aneurysm data sets, we assessed objective measures (accuracy and required time) and subjective ratings (suitability and likeability). The provided guidelines and presentation of different stages of the evaluation allow for an easy adaptation to other application areas of medical visualization.