RMM: A Recursive Mental Model for Dialogue Navigation

Roman, Homero Roman; Bisk, Yonatan; Thomason, Jesse; Çelikyılmaz, Aslı; Gao, Jianfeng

doi:10.18653/v1/2020.findings-emnlp.157

Cited by 25 publications

(26 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Spatial reasoning is a cognitive process based on the construction of mental representations for spatial objects, relations, and transformations (Clements and Battista, 1992), which is necessary for many natural language understanding (NLU) tasks such as natural language navigation Roman Roman et al, 2020;Kim et al, 2020), human-machine interaction (Landsiedel et al, 2017;Roman Roman et al, 2020), dialogue systems (Udagawa et al, 2020), and clinical analysis (Datta and Roberts, 2020).…”

Section: Introductionmentioning

confidence: 99%

SPARTQA: A Textual Question Answering Benchmark for Spatial Reasoning

Mirzaee¹,

Faghihi²,

Ning³

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

This paper proposes a question-answering (QA) benchmark for spatial reasoning on natural language text which contains more realistic spatial phenomena not covered by prior work and is challenging for state-of-the-art language models (LM). We propose a distant supervision method to improve on this task. Specifically, we design grammar and reasoning rules to automatically generate a spatial description of visual scenes and corresponding QA pairs. Experiments show that further pretraining LMs on these automatically generated data significantly improves LMs' capability on spatial understanding, which in turn helps to better solve two external datasets, bAbI, and boolQ. We hope that this work can foster investigations into more sophisticated models for spatial reasoning over text.

show abstract

Section: Introductionmentioning

confidence: 99%

SPARTQA: A Textual Question Answering Benchmark for Spatial Reasoning

Mirzaee¹,

Faghihi²,

Ning³

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

show abstract

“…Future work could extend WHY for dialogue (e.g., to clarify confusion or guide navigation (Roman et al, 2020)). This could incorporate natural language generation with deep learning and facilitate queries to the person.…”

Section: Discussionmentioning

confidence: 99%

Plan Explanations that Exploit a Cognitive Spatial Model

Korpan¹,

Epstein²

2021

Proceedings of Second International Combined Workshop on Spatial Language Understanding and Grounded Communication for Robotics

View full text Add to dashboard Cite

Ideally, people who navigate together in a complex indoor space share a mental model that facilitates explanation. This paper reports on a robot control system whose cognitive world model is based on spatial affordances that generalize over its perceptual data. Given a target, the control system formulates multiple plans, each with a model-relevant metric, and selects among them. As a result, it can provide readily understandable natural language about the robot's intentions and confidence, and generate diverse, contrastive explanations that reference the acquired spatial model. Empirical results in large, complex environments demonstrate the robot's ability to provide humanfriendly explanations in natural language.

show abstract

“…Recent work on dialog policy learning has employed theory of mind with a focus on specific domains. The Recursive Mental Model (RMM) (Roman et al, 2020) was proposed for navigation settings, where questions and answers are generated between a navigating agent and a guiding agent. Another approach -Answerer in Questioner's Mind (AQM) (Lee et al, 2018) -tackled an answer guessing game with information-theoretic methods.…”

Section: Related Workmentioning

confidence: 99%

Improving Dialog Systems for Negotiation with Personality Modeling

Yang¹,

Chen²,

Narasimhan³

2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

In this paper, we explore the ability to model and infer personality types of opponents, predict their responses, and use this information to adapt a dialog agent's high-level strategy in negotiation tasks. Inspired by the idea of incorporating a theory of mind (ToM) into machines, we introduce a probabilistic formulation to encapsulate the opponent's personality type during both learning and inference. We test our approach on the CRAIGSLISTBAR-GAIN dataset (He et al., 2018) and show that our method using ToM inference achieves a 20% higher dialog agreement rate compared to baselines on a mixed population of opponents. We also find that our model displays diverse negotiation behavior with different types of opponents. 1

show abstract

RMM: A Recursive Mental Model for Dialogue Navigation

Cited by 25 publications

References 37 publications

SPARTQA: A Textual Question Answering Benchmark for Spatial Reasoning

SPARTQA: A Textual Question Answering Benchmark for Spatial Reasoning

Plan Explanations that Exploit a Cognitive Spatial Model

Improving Dialog Systems for Negotiation with Personality Modeling

Contact Info

Product

Resources

About