Grounding spatial relations in natural language by fuzzy representation for human-robot interaction

Tan, Jiacheng; Ju, Zhaojie; Liu, Honghai

doi:10.1109/fuzz-ieee.2014.6891797

Cited by 13 publications

(4 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Space grounding: There are efforts mapping spatial relations to the region of actions using various representations: potential fields (Stopp et al 1994), discrete regions with fuzzy memberships (Tan, Ju, and Liu 2014), and points from a multi-class logistic classifier (Guadarrama et al 2013).…”

Section: Related Workmentioning

confidence: 99%

LINGO-Space: Language-Conditioned Incremental Grounding for Space

Kim,

Oh,

Hwang

et al. 2024

AAAI

View full text Add to dashboard Cite

We aim to solve the problem of spatially localizing composite instructions referring to space: space grounding. Compared to current instance grounding, space grounding is challenging due to the ill-posedness of identifying locations referred to by discrete expressions and the compositional ambiguity of referring expressions. Therefore, we propose a novel probabilistic space-grounding methodology (LINGO-Space) that accurately identifies a probabilistic distribution of space being referred to and incrementally updates it, given subsequent referring expressions leveraging configurable polar distributions. Our evaluations show that the estimation using polar distributions enables a robot to ground locations successfully through 20 table-top manipulation benchmark tests. We also show that updating the distribution helps the grounding method accurately narrow the referring space. We finally demonstrate the robustness of the space grounding with simulated manipulation and real quadruped robot navigation tasks. Code and videos are available at https://lingo-space.github.io.

show abstract

Section: Related Workmentioning

confidence: 99%

LINGO-Space: Language-Conditioned Incremental Grounding for Space

Kim,

Oh,

Hwang

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…Spatial relations have been used to enrich language-based human-robot interaction. Many works focus on using spatial relations to resolve referring expressions identifying objects ( Tan et al., 2014 ; Bao et al., 2016 ) as well as locations ( Tellex et al., 2011 ; Fasola and Matarić, 2013 ) for manipulation and navigation. In these works, the robot passively tries to parse the given command or description without querying the user for more information if necessary.…”

Section: Related Workmentioning

confidence: 99%

Interactive and incremental learning of spatial object relations from human demonstrations

Kartmann

Asfour

2023

Front. Robot. AI

View full text Add to dashboard Cite

Humans use semantic concepts such as spatial relations between objects to describe scenes and communicate tasks such as “Put the tea to the right of the cup” or “Move the plate between the fork and the spoon.” Just as children, assistive robots must be able to learn the sub-symbolic meaning of such concepts from human demonstrations and instructions. We address the problem of incrementally learning geometric models of spatial relations from few demonstrations collected online during interaction with a human. Such models enable a robot to manipulate objects in order to fulfill desired spatial relations specified by verbal instructions. At the start, we assume the robot has no geometric model of spatial relations. Given a task as above, the robot requests the user to demonstrate the task once in order to create a model from a single demonstration, leveraging cylindrical probability distribution as generative representation of spatial relations. We show how this model can be updated incrementally with each new demonstration without access to past examples in a sample-efficient way using incremental maximum likelihood estimation, and demonstrate the approach on a real humanoid robot.

show abstract

“…Many data-driven approaches for spatial relations have been developed in robotics, including applications in navigation [7], object manipulation [8,18,9] and human-robot interaction [10,24,11,25,26]. Zeng et al [9] designed a robot to manipulate objects given visual observations of the initial state and the goal.…”

Section: Introductionmentioning

confidence: 99%

Rel3D: A Minimally Contrastive Benchmark for Grounding Spatial Relations in 3D

Goyal¹,

Yang²,

Deng³

2020

Preprint

View full text Add to dashboard Cite

Understanding spatial relations (e.g., "laptop on table") in visual input is important for both humans and robots. Existing datasets are insufficient as they lack largescale, high-quality 3D ground truth information, which is critical for learning spatial relations. In this paper, we fill this gap by constructing Rel3D: the first large-scale, human-annotated dataset for grounding spatial relations in 3D. Rel3D enables quantifying the effectiveness of 3D information in predicting spatial relations on large-scale human data. Moreover, we propose minimally contrastive data collection-a novel crowdsourcing method for reducing dataset bias. The 3D scenes in our dataset come in minimally contrastive pairs: two scenes in a pair are almost identical, but a spatial relation holds in one and fails in the other. We empirically validate that minimally contrastive examples can diagnose issues with current relation detection models as well as lead to sample-efficient training. Code and data are available at https://github.com/princeton-vl/Rel3D.

show abstract

Grounding spatial relations in natural language by fuzzy representation for human-robot interaction

Cited by 13 publications

References 15 publications

LINGO-Space: Language-Conditioned Incremental Grounding for Space

LINGO-Space: Language-Conditioned Incremental Grounding for Space

Interactive and incremental learning of spatial object relations from human demonstrations

Rel3D: A Minimally Contrastive Benchmark for Grounding Spatial Relations in 3D

Contact Info

Product

Resources

About