Accuracy and Reliability Assessment of CT and MR Perfusion Analysis Software Using a Digital Phantom

Abstract

For the Stroke Imaging Repository (STIR) InvestigatorsPurpose:To design a digital phantom data set for computed tomography (CT) perfusion and perfusion-weighted imaging on the basis of the widely accepted tracer kinetic theory in which the true values of cerebral blood flow (CBF), cerebral blood volume (CBV), mean transit time (MTT), and tracer arrival delay are known and to evaluate the accuracy and reliability of postprocessing programs using this digital phantom. Materials and Methods:A phantom data set was created by generating concentration-time curves reflecting true values for CBF (2.5-87.5 mL/100 g per minute), CBV (1.0-5.0 mL/100 g), MTT (3.4-24 seconds), and tracer delays (0-3.0 seconds). These curves were embedded in human brain images. The data were analyzed by using 13 algorithms each for CT and magnetic resonance (MR), including five commercial vendors and five academic programs. Accuracy was assessed by using the Pearson correlation coefficient (r) for true values. Delay-, MTT-, or CBV-dependent errors and correlations between time to maximum of residue function (T max ) were also evaluated. Results:In CT, CBV was generally well reproduced (r . 0.9 in 12 algorithms), but not CBF and MTT (r . 0.9 in seven and four algorithms, respectively). In MR, good correlation (r . 0.9) was observed in one-half of commercial programs, while all academic algorithms showed good correlations for all parameters. Most algorithms had delay-dependent errors, especially for commercial software, as well as CBV dependency for CBF or MTT calculation and MTT dependency for CBV calculation. Correlation was good in T max except for one algorithm. Conclusion:The digital phantom readily evaluated the accuracy and characteristics of the CT and MR perfusion analysis software.

Exposing Hidden Truncation-Related Errors in Acute Stroke Perfusion Imaging

“…Clinical studies often omit consideration of the postprocessing algorithms that convert concentration measurements to perfusion maps, implicitly accepting their accuracy. However, postprocessing algorithms can produce significant errors, 5 which typically are visually undetectable during interpretation.…”

Performance and Predictive Value of a User-Independent Platform for CT Perfusion Analysis: Threshold-Derived Automated Systems Outperform Examiner-Driven Approaches in Outcome Prediction of Acute Ischemic Stroke

“…17 The relative performance of several tools was compared in a digital phantom in a recent work by Kudo et al 17 ; with this in mind, a study of predictive performance against commonly used qualitative approaches was conducted, suggesting enhancement to performance with the use of a standardized, user-independent platform. The disparate outcomes following re-analysis of data from the Echoplanar Imaging Thrombolytic Evaluation Trial support the idea that such differences in analysis may not be purely trivial.…”

“…10,11 Expert consensus has emphasized the demand for standardization in the acquisition, processing, and analysis of perfusion imaging. 15,16 The potential for disparate results and the variability in accuracy among competing software platforms have been the subject of recent studies and were thoroughly expounded in a comparative analysis by Kudo et al 17 In light of recent reports, the primary objective of this study were to examine the predictive performance of several user-defined approaches to NCCT, CTA, and CTP analysis, by comparison with a fast, vendor-and operator-independent computational tool using fully automated lesion segmentation and pixel-wise parametric thresholding for semiquantitation (RApid processing of PerfusIon and Diffusion [RAPID]). 18 The objective of our study was to evaluate these tools to determine their ability to predict 90-day favorable clinical outcome in patients with AIS.…”

Performance of CT ASPECTS and Collateral Score in Risk Stratification: Can Target Perfusion Profiles Be Predicted without Perfusion Imaging?

“…As a semi-quantitative CTP computing tool, RAPID has been shown to perform well, matching or exceeding the accuracy of similar software environments relative to a ground truth digital perfusion phantom in a recent study, and the use of similar iterations of the RAPID tool in recent multicenter trials may further support the generalizability of our findings. 3,4,5,7,27 Patient selection criteria likely modulate success in achieving a favorable clinical response following revascularization in acute ischemic stroke. The era of contemporary revascularization technologies now permits timely and dependable restoration of flow in most cases; however, optimal identification of a target population for treatment remains critical, and the ideal selection strategy remains inconclusively established.…”