Conversational AI : Open Domain Question Answering and Commonsense Reasoning

Basu, Kinjal

doi:10.4204/eptcs.306.53

Cited by 9 publications

(5 citation statements)

References 17 publications

(25 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Personalization has become an increasingly important part of these social systems. IrisBot, Genuine2, and Caspr use repeat user detection to slightly alter some responses [2,3,34]. Sound-ingBoard associated their users with Big-5 personality traits and adapted the conversation to those traits [12].…”

Section: Related Workmentioning

confidence: 99%

“…After filtering out low-quality and extraneous topics there are 635 pairs across 14 topics. We also annotate our data for kid-friendliness, which yields 342 question/answer pairs spanning all 14 supported topics marked as kid-friendly 3 .…”

Section: Data Collectionmentioning

confidence: 99%

“…The wouldyourather and hypotheticalsituation Reddit communities also have 316k and 102k subscribers respectively. 2 E.g., https://psycatgames.com/app/would-you-rather/ 3. Data available from https://nlds.soe.ucsc.edu/software.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Let's Get Personal: Personal Questions Improve SocialBot Performance in the Alexa Prize

Bowden¹,

Walker²

2023

Preprint

View full text Add to dashboard Cite

There has been an increased focus on creating conversational opendomain dialogue systems in the spoken dialogue community. Unlike traditional dialogue systems, these conversational systems cannot assume any specific information need or domain restrictions, i.e., the only inherent goal is to converse with the user on an unknown set of topics. While massive improvements in Natural Language Understanding (NLU) and the growth of available knowledge resources can partially support a robust conversation, these conversations generally lack the rapport between two humans that know each other. We developed a robust open-domain conversational system, Athena, that real Amazon Echo users access and evaluate at scale in the context of the Alexa Prize competition. We experiment with methods intended to increase intimacy between Athena and the user by heuristically developing a rule-based user model that personalizes both the current and subsequent conversations and evaluating specific personal opinion question strategies in A/B studies. Our results show a statistically significant positive impact on perceived conversation quality and length when employing these strategies.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Data Collectionmentioning

confidence: 99%

See 1 more Smart Citation

Let's Get Personal: Personal Questions Improve SocialBot Performance in the Alexa Prize

Bowden¹,

Walker²

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…There is also the widely held belief that, for a ‘general AI’ to truly emerge, commonsense reasoning is one problem, among others, that will need to be solved in a sufficiently robust manner (Baroni et al, 2017). A more functional reason for increased interest in commonsense reasoning is the rise of chatbots and other such ‘conversational AI’ services (e.g., Siri and Alexa) that represent an important area of innovation in industry (Basu, 2019; Gao et al, 2018; Ram et al, 2018; Young et al, 2017). Recently, the US Department of Defence also launched a machine common sense (MCS) program in which a diverse set of researchers and organizations, including the Allen Institute of Artificial Intelligence, is involved (Sap, Le Bras, et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

An experimental study measuring the generalization of fine‐tuned language representation models across commonsense reasoning benchmarks

Shen

Kejriwal

2023

Expert Systems

View full text Add to dashboard Cite

In the last 5 years, language representation models, such as BERT and GPT-3, based on transformer neural networks, have led to enormous progress in natural language processing (NLP). One such NLP task is commonsense reasoning, where performance is usually evaluated through multiple-choice question answering benchmarks. Till date, many such benchmarks have been proposed, and 'leaderboards' tracking stateof-the-art performance on those benchmarks suggest that transformer-based models are approaching human-like performance. Because these are commonsense benchmarks, however, such a model should be expected to generalize, that is, at least in aggregate, should not exhibit excessive performance loss across independent commonsense benchmarks regardless of the specific benchmark on (the training set of) which it has been fine-tuned. In this article, we evaluate this expectation by proposing a methodology and experimental study to measure the generalization ability of language representation models using a rigorous and intuitive metric. Using five established commonsense reasoning benchmarks, our experimental study shows that the models do not generalize well, and may be (potentially) susceptible to issues such as dataset bias. The results therefore suggest that current performance on benchmarks may be an over-estimate, especially if we want to use such models on novel commonsense problems for which a 'training' dataset may not be available, for the language representation model, to fine-tune on.

show abstract

“…Going a step further, we leverage SQuARE to build a general purpose closed-domain goal-oriented chatbot framework -StaCACK (pronounced as stack). Our work reported here builds upon our prior work in natural language QA as well as visual QA (Pendharkar and Gupta 2019;Basu 2019;.…”

Section: Introductionmentioning

confidence: 99%

Knowledge-driven Natural Language Understanding of English Text and its Applications

Basu¹,

Varanasi²,

Shakerin³

et al. 2021

AAAI

Self Cite

View full text Add to dashboard Cite

Understanding the meaning of a text is a fundamental challenge of natural language understanding (NLU) research. An ideal NLU system should process a language in a way that is not exclusive to a single task or a dataset. Keeping this in mind, we have introduced a novel knowledge driven semantic representation approach for English text. By leveraging the VerbNet lexicon, we are able to map syntax tree of the text to its commonsense meaning represented using basic knowledge primitives. The general purpose knowledge represented from our approach can be used to build any reasoning based NLU system that can also provide justification. We applied this approach to construct two NLU applications that we present here: SQuARE (Semantic-based Question Answering and Reasoning Engine) and StaCACK (Stateful Conversational Agent using Commonsense Knowledge). Both these systems work by ``truly understanding'' the natural language text they process and both provide natural language explanations for their responses while maintaining high accuracy.

show abstract

Conversational AI : Open Domain Question Answering and Commonsense Reasoning

Cited by 9 publications

References 17 publications

Let's Get Personal: Personal Questions Improve SocialBot Performance in the Alexa Prize

Let's Get Personal: Personal Questions Improve SocialBot Performance in the Alexa Prize

An experimental study measuring the generalization of fine‐tuned language representation models across commonsense reasoning benchmarks

Knowledge-driven Natural Language Understanding of English Text and its Applications

Contact Info

Product

Resources

About