Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering

Siriwardhana, Shamane; Weerasekera, Rivindu; Wen, Elliott; Kaluarachchi, Tharindu; Rana, Rajib; Nanayakkara, Suranga

doi:10.1162/tacl_a_00530

Cited by 24 publications

(13 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The aim of the retrieval agent is to take a query and respond with a concise answer from the user's encoded external memories, enabling the Query Mode. It uses a method called retrieval augmented generation developed by Lewis et al [42] and used in state of the art question-answering systems [50,65].…”

Section: Retrieval Agentmentioning

confidence: 99%

Memoro: Using Large Language Models to Realize a Concise Interface for Real-Time Memory Augmentation

Zulfikar,

Chan,

Maes

2024

Proceedings of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

People have to remember an ever-expanding volume of information. Wearables that use information capture and retrieval for memory augmentation can help but can be disruptive and cumbersome in real-world tasks, such as in social settings. To address this, we developed Memoro, a wearable audio-based memory assistant with a concise user interface. Memoro uses a large language model (LLM) to infer the user's memory needs in a conversational context, semantically search memories, and present minimal suggestions. The assistant has two interaction modes: Query Mode for voicing queries and Queryless Mode for on-demand predictive assistance, without explicit query. Our study of (N=20) participants engaged in a realtime conversation, demonstrated that using Memoro reduced device Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).

show abstract

Section: Retrieval Agentmentioning

confidence: 99%

Memoro: Using Large Language Models to Realize a Concise Interface for Real-Time Memory Augmentation

Zulfikar,

Chan,

Maes

2024

Proceedings of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

show abstract

“…This was further elaborated upon through research demonstrating RAG's capacity to improve factual accuracy and relevance in responses by leveraging up-to-date external databases [38], [39]. The effectiveness of RAG in domain-specific applications was analyzed, showing marked improvements in areas requiring specialized knowledge, such as scientific research and historical facts [40], [41]. Comparative studies between RAG and traditional LLMs underscored the former's superior performance in tasks demanding detailed, accurate information [42]- [44].…”

Section: Ragmentioning

confidence: 99%

Merging Mixture of Experts and Retrieval Augmented Generation for Enhanced Information Retrieval and Reasoning

Xiong,

Zheng

2024

Preprint

View full text Add to dashboard Cite

This study investigates the integration of Retrieval Augmented Generation (RAG) into the Mistral 8x7B Large Language Model (LLM), which already uses Mixture of Experts (MoE), to address its existing limitations in complex information retrieval and reasoning tasks. By leveraging the Google BIG-Bench dataset, we conducted extensive quantitative and qualitative analyses to evaluate the augmented model's performance. The results demonstrate significant improvements in accuracy, precision, recall, and F1 score, highlighting the enhanced model's superior capability to generate contextually rich, accurate, and nuanced responses. This integration showcases a promising approach to overcoming the intrinsic limitations of traditional LLMs, marking a pivotal advancement in artificial intelligence research. Our findings contribute to the ongoing development of more adaptable, efficient, and intelligent AI systems, opening new avenues for AI application across various fields. The study acknowledges limitations related to dataset scope and computational demands, suggesting directions for future research to further refine and expand the model's applicability.

show abstract

“…This retrieved context is subsequently integrated into a generation model, such as a large language model to augment its capabilities in generating responses or completing tasks. Despite the widespread applications of RAG across various tasks including question-answering [35], RAG has not been directly utilized in the task of question retrieval in CQA. Our research aligns with both the first and third categories, leveraging metadata to improve question retrieval in community question answering.…”

Section: Related Workmentioning

confidence: 99%

Harnessing the Power of Metadata for Enhanced Question Retrieval in Community Question Answering

Ghasemi,

Shakery

2024

IEEE Access

View full text Add to dashboard Cite

Community Question Answering (CQA) forums such as Yahoo! Answers and Stack Overflow have become popular. The main goal of a CQA is to provide the most suitable answer in the shortest possible time. Since there is a reach archive of answered questions, similar question retrieval has received much attention intending to answer questions immediately after asking. One of the main challenges in this task is the lexical gap between questions, which refers to the discrepancies between the terminologies used by users asking questions. In this paper, we use metadata and two transformer-based techniques to improve the translation-based language model as a traditional technique addressing the lexical gap in retrieval systems. To overcome the lexical gap problem, additional context and information about the questions can help. Metadata is a rich source of information that refers to supplementary data associated with each question. Subject, category, and answer are metadata used in this article. To leverage these metadata, two transformerbased methods are employed. First, to utilize category information, we build category-specific dictionaries to obtain more accurate translation probabilities. A BERT model predicts the categories of the questions. Second, to utilize answer information, we propose a question expansion technique. Expansion is done by a transformer-based model using a retrieval-augmented generation (RAG) model to generate answers and expand new questions with corresponding answers. Finally, candidate questions are ranked according to their similarity to the expanded new question. Our proposed method achieves 51.47 in terms of MAP, outperforming all state-of-the-art approaches in question retrieval.

show abstract

Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering

Cited by 24 publications

References 38 publications

Memoro: Using Large Language Models to Realize a Concise Interface for Real-Time Memory Augmentation

Memoro: Using Large Language Models to Realize a Concise Interface for Real-Time Memory Augmentation

Merging Mixture of Experts and Retrieval Augmented Generation for Enhanced Information Retrieval and Reasoning

Harnessing the Power of Metadata for Enhanced Question Retrieval in Community Question Answering

Contact Info

Product

Resources

About