Assessing a Single Image in Reference-Guided Image Synthesis

Abstract

Assessing the performance of Generative Adversarial Networks (GANs) has been an important topic due to its practical significance. Although several evaluation metrics have been proposed, they generally assess the quality of the whole generated image distribution. For Reference-guided Image Synthesis (RIS) tasks, i.e., rendering a source image in the style of another reference image, where assessing the quality of a single generated image is crucial, these metrics are not applicable. In this paper, we propose a general learning-based framework, Reference-guided Image Synthesis Assessment (RISA) to quantitatively evaluate the quality of a single generated image. Notably, the training of RISA does not require human annotations. In specific, the training data for RISA are acquired by the intermediate models from the training procedure in RIS, and weakly annotated by the number of models' iterations, based on the positive correlation between image quality and iterations. As this annotation is too coarse as a supervision signal, we introduce two techniques: 1) a pixel-wise interpolation scheme to refine the coarse labels, and 2) multiple binary classifiers to replace a naïve regressor. In addition, an unsupervised contrastive loss is introduced to effectively capture the style similarity between a generated image and its reference image. Empirical results on various datasets demonstrate that RISA is highly consistent with human preference and transfers well across models.

A New Approach to Interior Design: Generating Creative Interior Design Videos of Various Design Styles from Indoor Texture-Free 3D Models

“…Previous studies indicate that objective assessment metrics cannot fully reflect human perception [49][50][51]. Therefore, instead of relying solely on accurate assessment metrics, we also conducted subjective assessments to enhance the credibility of the conclusions [49].…”

A therapeutic hepatitis B mRNA vaccine with strong immunogenicity and persistent virological suppression

“…To comprehensively evaluate Text-to-Image (T2I) models, there are currently various datasets available. Among them, the COCO validation set [20] is one of the most widely used datasets [7,11,12,23]. In addition, to enhance benchmark precision, Saharia et al [29] conduct the first attempt to consider multiple evaluation aspects and propose DrawBench, which contains 200 prompts specifically designed for evaluating T2I tasks.…”

Balanced Contrastive Learning for Long-Tailed Visual Recognition

“…However, implementing our idea is not straightforward due to two obstacles: 1) the methodology for modeling the actual data distribution is typically complex, e.g., training deep generative models [20], [21], [22]; and 2) calculating the expected training loss in a closed form is difficult. In this paper, we simultaneously tackle both challenges by proposing a novel probabilistic contrastive learning algorithm as illustrated in Fig.…”