Users on the internet usually have conversations on interesting facts or topics along with diverse knowledge from the web. However, most existing knowledge-grounded conversation models consider only a single document regarding the topic of a conversation. The recently proposed retrievalaugmented models generate a response based on multiple documents; however, they ignore the given topic and use only the local context of the conversation. To this end, we introduce a novel retrieval-augmented response generation model that retrieves an appropriate range of documents relevant to both the topic and local context of a conversation and uses them for generating a knowledge-grounded response. Our model first accepts both topic words extracted from the whole conversation and the tokens before the response to yield multiple representations. It then chooses representations of the first N token and ones of keywords from the conversation and document encoders and compares the two groups of representation from the conversation with those groups of the document, respectively. For training, we introduce a new dataweighting scheme to encourage the model to produce knowledge-grounded responses without ground truth knowledge. Both automatic and human evaluation results with a large-scale dataset show that our models can generate more knowledgeable, diverse, and relevant responses compared to the state-of-the-art models.
This paper is a fully rewritten extended version that features the Bernoulli and Gumbel methods and detailed analysis of the proposed model, including the essential hyperparameters as well as extensive experiments with new datasets and additional baseline models.
While recent advances of GAN models enabled photo-realistic synthesis of various object images, challenges still remain in modeling more complex image distributions such as scenes with multiple objects. The difficulty lies in the high structural complexity of scene images, where the discriminator carries a heavy burden in discriminating complex structural differences between real and fake scene images. Therefore, enhancing the discriminative capability of the discriminator could be one of the effective strategies to improve the generation performance of GAN models. In this paper, we explore ways to boost the discriminative capability by leveraging two recent paradigms on visual representation learning: selfsupervised learning and transfer learning. As the first approach, we propose a self-supervised auxiliary task tailored to enhance the multi-scale representations of the discriminator. In the second approach, we further enhance the discriminator by utilizing pretrained representations from various scene understanding models. To fully utilize knowledge from multiple expert models, we propose a multi-scale feature ensemble to mix multi-sale representations. Empirical results on challenging scene datasets demonstrate that the proposed strategies significantly advance the generation performance, enabling diverse and photo-realistic synthesis of complex scene images. a a This is an extended and revised version of a conference paper [1] that was presented in WACV 2023. In this paper, we introduce an additional approach that further improves the discriminator representations by utilizing pretrained expert models (Section V). We validate the improvement with additional experimental results and a corresponding ablation study (Section VI).
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.