AAPM Task Group Report 311: Guidance for performance evaluation of fluorescence‐guided surgery systems

Abstract

The last decade has seen a large growth in fluorescence‐guided surgery (FGS) imaging and interventions. With the increasing number of clinical specialties implementing FGS, the range of systems with radically different physical designs, image processing approaches, and performance requirements is expanding. This variety of systems makes it nearly impossible to specify uniform performance goals, yet at the same time, utilization of different devices in new clinical procedures and trials indicates some need for common knowledge bases and a quality assessment paradigm to ensure that effective translation and use occurs. It is feasible to identify key fundamental image quality characteristics and corresponding objective test methods that should be determined such that there are consistent conventions across a variety of FGS devices. This report outlines test methods, tissue simulating phantoms and suggested guidelines, as well as personnel needs and professional knowledge bases that can be established. This report frames the issues with guidance and feedback from related societies and agencies having vested interest in the outcome, coming from an independent scientific group formed from academics and international federal agencies for the establishment of these professional guidelines.

Impact of signal-to-noise ratio and contrast definition on the sensitivity assessment and benchmarking of fluorescence molecular imaging systems

“…In a recent report, the American Association of Physicists in Medicine (AAPM) proposed SNR 4 and , as metrics for the performance assessment of fluorescence imaging systems. 17 Moreover, the suggested background region proposed in these guidelines for the estimation of SNR corresponds to a region with the same optical properties as the interrogated wells, but without fluorescent dye. This corresponds to the ROI b 2 in our study since the wells are gradually covered with the phantom matrix material which has no fluorescent dye.…”

“… 19 Thus, the reference threshold values applied here are user-independent in comparison to another value commonly used in fluorescence imaging, the Rose criterion. 17 The Rose criterion method also sets a limit of detection for fluorescence imaging for which the CNR values must be greater than 3 to 5. 40 However, the range of a particular threshold value varies from study to study 35 , 41 , 42 and depends on several parameters such as object shape, edge sharpness, viewing distance, and observer experience.…”

“…Over the past few years, numerous studies on phantom development and standardization procedures, as well as attempts to achieve consensus in the community, have been reported. 1 , 2 , 9 , 11 – 19 Thus far, methods and reference targets for system evaluation and comparison have been developed on an individual basis, but a universal cross-platform metric for image fidelity evaluation has yet to be developed. 16 …”

“…Currently, the sensitivity of FMI systems is assessed mostly using the signal-to-noise ratio (SNR) and/or contrast metrics. 2 , 12 , 17 , 20 It has been shown, however, that the definition of the background can play a significant role in the interpretation of the acquired images, especially during tissue imaging. 21 – 24 For example, Chen et al.…”

“…In specifics, using six near-infrared FMI systems, we captured fluorescence images of a composite rigid phantom previously developed by our group. 11 , 18 , 19 We then assessed the sensitivity 19 of those systems using six previously published formulas for SNR and contrast 17 , 29 – 33 and two background locations. Moreover, based on these metrics, we quantified the corresponding benchmarking (BM) scores, 19 and the systems were ranked based on these scores.…”