Robotic Understanding of Spatial Relationships Using Neural-Logic Learning

Yan, Fujian; Wang, Dali; He, Hongsheng

doi:10.1109/iros45743.2020.9340917

Cited by 11 publications

(5 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…to a predicate (e.g., IN(apple, kitchencabinet)), indicating the spatial relationship between two entities. Such representations have been widely adopted in robotics and embodied AI [38]- [41]. Given the particles, the helping planner assesses the value of the edges in the intermediate states and the final goals and selects the most valuable edge as the helping subgoal.…”

Section: B Methods Overviewmentioning

confidence: 99%

NOPA: Neurally-guided Online Probabilistic Assistance for Building Socially Intelligent Home Assistants

Puig¹,

Shu²,

Tenenbaum³

et al. 2023

Preprint

View full text Add to dashboard Cite

In this work, we study how to build socially intelligent robots to assist people in their homes. In particular, we focus on assistance with online goal inference, where robots must simultaneously infer humans' goals and how to help them achieve those goals. Prior assistance methods either lack the adaptivity to adjust helping strategies (i.e., when and how to help) in response to uncertainty about goals or the scalability to conduct fast inference in a large goal space. Our NOPA (Neurally-guided Online Probabilistic Assistance) method addresses both of these challenges. NOPA consists of (1) an online goal inference module combining neural goal proposals with inverse planning and particle filtering for robust inference under uncertainty, and (2) a helping planner that discovers valuable subgoals to help with and is aware of the uncertainty in goal inference. We compare NOPA against multiple baselines in a new embodied AI assistance challenge: Online Watch-And-Help, in which a helper agent needs to simultaneously watch a main agent's action, infer its goal, and help perform a common household task faster in realistic virtual home environments. Experiments show that our helper agent robustly updates its goal inference and adapts its helping plans to the changing level of uncertainty. 1

show abstract

Section: B Methods Overviewmentioning

confidence: 99%

NOPA: Neurally-guided Online Probabilistic Assistance for Building Socially Intelligent Home Assistants

Puig¹,

Shu²,

Tenenbaum³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Many works have introduced models to classify existing spatial relations between objects to improve scene understanding ( Rosman and Ramamoorthy, 2011 ; Sjöö and Jensfelt, 2011 ; Fichtl et al., 2014 ; Yan et al., 2020 ) and human activity recognition ( Zampogiannis et al., 2015 ; Dreher et al., 2020 ; Lee et al., 2020 ). These models are either hand-crafted or are not learned incrementally.…”

Section: Related Workmentioning

confidence: 99%

Interactive and incremental learning of spatial object relations from human demonstrations

Kartmann

Asfour

2023

Front. Robot. AI

View full text Add to dashboard Cite

Humans use semantic concepts such as spatial relations between objects to describe scenes and communicate tasks such as “Put the tea to the right of the cup” or “Move the plate between the fork and the spoon.” Just as children, assistive robots must be able to learn the sub-symbolic meaning of such concepts from human demonstrations and instructions. We address the problem of incrementally learning geometric models of spatial relations from few demonstrations collected online during interaction with a human. Such models enable a robot to manipulate objects in order to fulfill desired spatial relations specified by verbal instructions. At the start, we assume the robot has no geometric model of spatial relations. Given a task as above, the robot requests the user to demonstrate the task once in order to create a model from a single demonstration, leveraging cylindrical probability distribution as generative representation of spatial relations. We show how this model can be updated incrementally with each new demonstration without access to past examples in a sample-efficient way using incremental maximum likelihood estimation, and demonstrate the approach on a real humanoid robot.

show abstract

“…There is a huge requirement for correct recognition between similar traits [5] of two different spaces. According to Yan et al [7], understanding spatial relations of objects is critical in many robotic applications such as grasping, manipulation, and obstacle avoidance. On the other hand, Humans can simply reason about an object's spatial relations from a glimpse of a scene based on prior knowledge of spatial constraints.…”

Section: Introductionmentioning

confidence: 99%

“…Several indoor scene recognition techniques including robots have been presented in recent years [5,[7][8][9][10][11][12][13][14]. RGB-D data is utilized to detect object orientation in many existing recognition methods, and it is incorporated with the definition of objects in the scene [15,16].…”

Section: Introductionmentioning

confidence: 99%

Reliable Scene Recognition Approach for Mobile Robots with Limited Resources Based on Deep Learning and Neuro-Fuzzy Inference

Singh¹,

Pandey²,

Puig³

et al. 2022

View full text Add to dashboard Cite

Indoor scene recognition is complex due to the commonality shared between different spaces. Still, when it comes to robotics applications, the uncertainty increases due to illumination change, motion blur, interruption due to external light sources, and cluttered environments. Most existing fusion approaches do not consider the uncertainty, and others have a high computational cost that may not suit robots with limited resources. To mitigate these issues, this paper proposes a reliable indoor scene recognition approach for mobile robots with limited resources based on robust deep convolutional neural networks (CNNs) feature extractors and neuro-fuzzy inference to consider the uncertainty of the data. All CNN feature extractors are pre-trained on the Imagenet dataset and used in the manner of transfer learning. The performance of our fusion method has been assessed on a customized MIT-67 dataset and for real-time processing on a Locobot robot. We also compare the proposed method with two standard fusion methods---Early Feature Fusion (EFF) and Weighted Average Late Fusion (WALF). The experimental results demonstrate that our method achieves competitive results with a precision of 94%, and it performs well on the Locobot robot with a speed of 3.1 frames per second.

show abstract

Robotic Understanding of Spatial Relationships Using Neural-Logic Learning

Cited by 11 publications

References 33 publications

NOPA: Neurally-guided Online Probabilistic Assistance for Building Socially Intelligent Home Assistants

NOPA: Neurally-guided Online Probabilistic Assistance for Building Socially Intelligent Home Assistants

Interactive and incremental learning of spatial object relations from human demonstrations

Reliable Scene Recognition Approach for Mobile Robots with Limited Resources Based on Deep Learning and Neuro-Fuzzy Inference

Contact Info

Product

Resources

About