Validation and evaluation metrics for medical and biomedical image synthesis

Nečasová, Tereza; Burgos, Ninon; Svoboda, David

doi:10.1016/b978-0-12-824349-7.00032-3

Cited by 6 publications

(7 citation statements)

References 76 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In addition, four metrics were applied: the mean absolute error (MAE) or L1 norm of the error, the normalized root mean squared error (NRMSE), the structural similarity index measure (SSIM), and the peak-signal-to-noise-ratio (PSNR). These metrics are standard performance measures for pairwise image comparison and were selected to better compare with existing studies ( 30 ).…”

Section: Methodsmentioning

confidence: 99%

Image-to-image generative adversarial networks for synthesizing perfusion parameter maps from DSC-MR images in cerebrovascular disease

et al. 2023

View full text Add to dashboard Cite

Stroke is a major cause of death or disability. As imaging-based patient stratification improves acute stroke therapy, dynamic susceptibility contrast magnetic resonance imaging (DSC-MRI) is of major interest in image brain perfusion. However, expert-level perfusion maps require a manual or semi-manual post-processing by a medical expert making the procedure time-consuming and less-standardized. Modern machine learning methods such as generative adversarial networks (GANs) have the potential to automate the perfusion map generation on an expert level without manual validation. We propose a modified pix2pix GAN with a temporal component (temp-pix2pix-GAN) that generates perfusion maps in an end-to-end fashion. We train our model on perfusion maps infused with expert knowledge to encode it into the GANs. The performance was trained and evaluated using the structural similarity index measure (SSIM) on two datasets including patients with acute stroke and the steno-occlusive disease. Our temp-pix2pix architecture showed high performance on the acute stroke dataset for all perfusion maps (mean SSIM 0.92–0.99) and good performance on data including patients with the steno-occlusive disease (mean SSIM 0.84–0.99). While clinical validation is still necessary for future studies, our results mark an important step toward automated expert-level perfusion maps and thus fast patient stratification.

show abstract

Section: Methodsmentioning

confidence: 99%

Image-to-image generative adversarial networks for synthesizing perfusion parameter maps from DSC-MR images in cerebrovascular disease

et al. 2023

View full text Add to dashboard Cite

show abstract

“…In [ 27 , 28 , 29 ], a review and deep analysis of the evaluation methods of synthetic images were given, and important analytical conclusions about their disadvantages and advantages were drawn. When generative models began to be used in healthcare, the simplest Image Quality Assessment methods were first used to evaluate the quality of the generated images.…”

Section: Related Workmentioning

confidence: 99%

Evaluating Synthetic Medical Images Using Artificial Intelligence with the GAN Algorithm

Abdusalomov

Nasimov

Nasimova

et al. 2023

Sensors

View full text Add to dashboard Cite

In recent years, considerable work has been conducted on the development of synthetic medical images, but there are no satisfactory methods for evaluating their medical suitability. Existing methods mainly evaluate the quality of noise in the images, and the similarity of the images to the real images used to generate them. For this purpose, they use feature maps of images extracted in different ways or distribution of images set. Then, the proximity of synthetic images to the real set is evaluated using different distance metrics. However, it is not possible to determine whether only one synthetic image was generated repeatedly, or whether the synthetic set exactly repeats the training set. In addition, most evolution metrics take a lot of time to calculate. Taking these issues into account, we have proposed a method that can quantitatively and qualitatively evaluate synthetic images. This method is a combination of two methods, namely, FMD and CNN-based evaluation methods. The estimation methods were compared with the FID method, and it was found that the FMD method has a great advantage in terms of speed, while the CNN method has the ability to estimate more accurately. To evaluate the reliability of the methods, a dataset of different real images was checked.

show abstract

“…The main challenges of pseudo-healthy reconstruction are to preserve the subject identity in the reconstructed image and to ensure that the model outputs are healthy brains. 9 To evaluate the quality of the reconstruction, four metrics are often used in the literature: 15 the mean absolute error (MAE), the mean squared error (MSE), the peak signal-to-noise ratio (PSNR) and the structural similarity index (SSIM). 16 But if these metrics can be used to measure the conservation of the subject identity, they are not suited to evaluate healthiness of the synthesized images.…”

Section: Simulation-based Evaluation Frameworkmentioning

confidence: 99%

Simulation based evaluation framework for deep learning unsupervised anomaly detection on brain FDG-PET

Hassanaly

Bottani

Sauty

et al. 2023

Medical Imaging 2023: Image Processing

Self Cite

View full text Add to dashboard Cite

Unsupervised anomaly detection using deep learning models is a popular computer-aided diagnosis approach because it does not need annotated data and is not restricted to the diagnosis of a disease seen during training. Such approach consists in first learning the distribution of anomaly free images. Images presenting anomalies are then detected as outliers of this distribution. These approaches have been widely applied in neuroimaging to detect sharp and localized anomalies such as tumors or white matter hyper-intensities from structural MRI. In this work, we aim to detect anomalies from FDG PET images of patients with Alzheimer's disease. In this context, the anomalies can be subtle and difficult to delineate, making the task more difficult and meaning that no ground truth exists to evaluate the approaches. We thus propose a framework to evaluate unsupervised anomaly detection approaches that consists in simulating realistic anomalies from images of healthy subjects. We demonstrate the use of this framework by evaluating an approach based on a 3D variational autoencoder.

show abstract

Validation and evaluation metrics for medical and biomedical image synthesis

Cited by 6 publications

References 76 publications

Image-to-image generative adversarial networks for synthesizing perfusion parameter maps from DSC-MR images in cerebrovascular disease

Image-to-image generative adversarial networks for synthesizing perfusion parameter maps from DSC-MR images in cerebrovascular disease

Evaluating Synthetic Medical Images Using Artificial Intelligence with the GAN Algorithm

Simulation based evaluation framework for deep learning unsupervised anomaly detection on brain FDG-PET

Contact Info

Product

Resources

About