The Validity of Using Roc Software for Analysing Visual Grading Characteristics Data: An Investigation Based on the Novel Software VGC Analyzer

Abstract

The purpose of the present work was to investigate the validity of using single-reader-adapted receiver operating characteristics (ROC) software for analysis of visual grading characteristics (VGC) data. VGC data from four published VGC studies on optimisation of X-ray examinations, previously analysed using ROCFIT, were reanalysed using a recently developed software dedicated to VGC analysis (VGC Analyzer), and the outcomes [the mean and 95 % confidence interval (CI) of the area under the VGC curve (AUCVGC) and the p-value] were compared. The studies included both paired and non-paired data and were reanalysed both for the fixed-reader and the random-reader situations. The results showed good agreement between the softwares for the mean AUCVGC For non-paired data, wider CIs were obtained with VGC Analyzer than previously reported, whereas for paired data, the previously reported CIs were similar or even broader. Similar observations were made for the p-values. The results indicate that the use of single-reader-adapted ROC software such as ROCFIT for analysing non-paired VGC data may lead to an increased risk of committing Type I errors, especially in the random-reader situation. On the other hand, the use of ROC software for analysis of paired VGC data may lead to an increased risk of committing Type II errors, especially in the fixed-reader situation.

Iterative scatter correction for grid-less skeletal radiography allows improved image quality equal to an antiscatter grid in adjunct with dose reduction: a visual grading study of 20 body donors

“…If AUC VGC is not significantly different from 0.5, the two conditions cannot be separated statistically. The software used for calculation was VGC Analyzer (18,19,21,22). This is a statistical tool for fully crossed multiple-reader multiple-case VGC studies.…”

Pulmonary artery imaging under free‐breathing using golden‐angle radial bSSFP MRI: a proof of concept

“…Analysis of the observer's scores was done using Visual Grading Characteristics (VGC) in the freely available VGC Analyzer software package . Visual Grading Characteristics is a nonparametric, rank, invariant method for statistical analysis of observer‐reported image criteria scores (ICS).…”

“…The ICS is a reflection of how well the image in question fulfills the image criterion, and the analysis assesses how the proportion of the test condition images that are scored higher than a certain threshold varies with the proportion of the reference condition, when the threshold is changed . A nonunitary ratio indicates a difference in confidence and an area under the cumulative distribution curve significantly greater than 0.5 indicates a significantly higher confidence in the test condition . In each image criteria class, the Cartesian trajectory was considered the reference condition, and the radial trajectory the test condition.…”

Evaluation of an iterative model-based reconstruction of pediatric abdominal CT with regard to image quality and radiation dose

“…The visibility of anatomic structures (questions A–E, Table 2) was evaluated with a visual grading characteristics (VGC) analysis (14,15). The VGC Analyzer software v1.0 (The Sahlgrenska Academy at University of Gothenburg, Gothenburg, Sweden), written in IDL (Research System, Inc, Boulder, CO, USA), was used to evaluate the results (16,17). The program produced a statistical analysis of the rating data from studies performed with multiple readers, where multiple cases were assessed, and all readers assessed all cases.…”