Building and Evaluating Open-Domain Dialogue Corpora with Clarifying Questions

Aliannejadi, Mohammad; Kiseleva, Julia; Chuklin, Aleksandr; Dalton, Jeff; Burtsev, Mikhail

doi:10.48550/arxiv.2109.05794

Cited by 2 publications

(3 citation statements)

References 37 publications

(50 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, the community has focused on continuous learning through interactions, including systems that learn a new task from instructions [57], assess their uncertainty [89] and ask feedback from humans in case of uncertainty [3,4] or for correcting possible mistakes [30].…”

Section: Background and Related Workmentioning

confidence: 99%

Supporting Complex Information-Seeking Tasks with Implicit Constraints

Ahmadvand¹,

Arabzadeh²,

Kiseleva³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Current interactive systems with natural language interface lack an ability to understand a complex information-seeking request which expresses several implicit constraints at once, and there is no prior information about user preferences, e.g., 'find hiking trails around San Francisco which are accessible with toddlers and have beautiful scenery in summer', where output is a list of possible suggestions for users to start their exploration. In such scenarios, the user requests can be issued at once in the form of a complex and long query, unlike conversational and exploratory search models that require short utterances or queries where they often require to be fed into the system step by step. This advancement provides the final user more flexibility and precision in expressing their intent through the search process. Such systems are inherently helpful for day-today user tasks requiring planning that are usually time-consuming, sometimes tricky, and cognitively taxing. We have designed and deployed a platform to collect the data from approaching such complex interactive systems.In this paper, we propose an Interactive Agent (IA) that allows intricately refined user requests by making it complete, which should lead to better retrieval. To demonstrate the performance of the proposed modeling paradigm, we have adopted various pre-retrieval metrics that capture the extent to which guided interactions with our system yield better retrieval results. Through extensive experimentation, we demonstrated that our method significantly outperforms several robust baselines.

show abstract

Section: Background and Related Workmentioning

confidence: 99%

Supporting Complex Information-Seeking Tasks with Implicit Constraints

Ahmadvand¹,

Arabzadeh²,

Kiseleva³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Recent efforts have also focused on interactivity and continuous learning to enable agents to interact with users to resolve the knowledge gap between them for better accuracy and transparency. This includes systems that can learn new task from instructions [Li et al, 2020], assess their uncertainty [Yao et al, 2019], ask clarifying questions [Aliannejadi et al, 2020[Aliannejadi et al, , 2021 and seek and leverage feedback from humans to correct mistakes [Elgohary et al, 2020].…”

Section: Competition Typementioning

confidence: 99%

“…There is a long history of competitions focused on NLU/G tasks. Especially in recent years we have seen a large number of challenges dedicated to open-domain dialog systems [Hauff et al, 2021, Dalton et al, 2020, Spina et al, 2019, Chuklin et al, 2018, such as ConvAI [Burtsev and Logacheva, 2020], ConvAI2 , ConvAI3: Clarifying Questions for Open-Domain Dialogue Systems (ClariQ) [Aliannejadi et al, 2020[Aliannejadi et al, , 2021, as well as a series of competitions of the Alexa Prize 5 . There are great efforts in the community to advance task-oriented dialogs by suggesting competitions, such as the Dialog System Technology Challenge (DSTC-8) [Kim et al, 2019]; benchmarks and experimental platforms, e.g., Convlab, which offers the annotated MultiWOZ dataset [Budzianowski et al, 2018] and associated pre-trained reference models .…”

Section: Noveltymentioning

confidence: 99%

NeurIPS 2021 Competition IGLU: Interactive Grounded Language Understanding in a Collaborative Environment

Kiseleva,

Li,

Aliannejadi

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Human intelligence has the remarkable ability to quickly adapt to new tasks and environments. Starting from a very young age, humans acquire new skills and learn how to solve new tasks either by imitating the behavior of others or by following provided natural language instructions. To facilitate research in this direction, we propose IGLU: Interactive Grounded Language Understanding in a Collaborative Environment.The primary goal of the competition is to approach the problem of how to build interactive agents that learn to solve a task while provided with grounded natural language instructions in a collaborative environment. Understanding the complexity of the challenge, we split it into sub-tasks to make it feasible for participants.This research challenge is naturally related, but not limited, to two fields of study that are highly relevant to the NeurIPS community: Natural Language Understanding and Generation (NLU/G) and Reinforcement Learning (RL). Therefore, the suggested challenge can bring two communities together to approach one of the important challenges in AI. Another important aspect of the challenge is the dedication to perform a human-in-the-loop evaluation as a final evaluation for the agents developed by contestants.

show abstract

Building and Evaluating Open-Domain Dialogue Corpora with Clarifying Questions

Cited by 2 publications

References 37 publications

Supporting Complex Information-Seeking Tasks with Implicit Constraints

Supporting Complex Information-Seeking Tasks with Implicit Constraints

NeurIPS 2021 Competition IGLU: Interactive Grounded Language Understanding in a Collaborative Environment

Contact Info

Product

Resources

About