Robotic Occlusion Reasoning for Efficient Object Existence Prediction

Li, Mengdi; Weber, Cornelius; Kerzel, Matthias; Lee, Jae Hee; Zeng, Zheni; Liu, Zhiyuan; Wermter, Stefan

doi:10.1109/iros51168.2021.9635947

Cited by 5 publications

(3 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A specific spatial relation between objects arises when one object occludes another, i.e., when one object is behind another from the observer's point of view. In the task of robotic object existence prediction by occlusion reasoning (Li et al, 2021 ), a robot needs to reason whether a target object is possibly occluded by a visible object. Curriculum learning has proven essential for the successful training of the proposed model.…”

Section: Discussionmentioning

confidence: 99%

“…While our models are trained in a supervised way from pairs of input and output vectors, interacting with the environment means that actions are iteratively performed by an embodied learning agent ( Figure 9 shows the environment in the loop). There are many approaches to train the action policy of an agent, including supervised learning (Shah et al, 2021 ), imitation learning (Chevalier-Boisvert et al, 2019 ; Chaplot et al, 2020 ; Shridhar et al, 2021 ), and reinforcement learning (Hermann et al, 2017 ; Chaplot et al, 2018 ; Li et al, 2021 ). Among these approaches, reinforcement learning is most versatile because it does not require human-labeled data for all situations, but the agent can learn its action policy by interacting with the environment and only occasionally receiving rewards (purple dashed arrow in Figure 9 ).…”

Section: Concept Of An Integrated Architecturementioning

confidence: 99%

See 1 more Smart Citation

Spatial relation learning in complementary scenarios with deep neural networks

et al. 2022

Self Cite

View full text Add to dashboard Cite

A cognitive agent performing in the real world needs to learn relevant concepts about its environment (e.g., objects, color, and shapes) and react accordingly. In addition to learning the concepts, it needs to learn relations between the concepts, in particular spatial relations between objects. In this paper, we propose three approaches that allow a cognitive agent to learn spatial relations. First, using an embodied model, the agent learns to reach toward an object based on simple instructions involving left-right relations. Since the level of realism and its complexity does not permit large-scale and diverse experiences in this approach, we devise as a second approach a simple visual dataset for geometric feature learning and show that recent reasoning models can learn directional relations in different frames of reference. Yet, embodied and simple simulation approaches together still do not provide sufficient experiences. To close this gap, we thirdly propose utilizing knowledge bases for disembodied spatial relation reasoning. Since the three approaches (i.e., embodied learning, learning from simple visual data, and use of knowledge bases) are complementary, we conceptualize a cognitive architecture that combines these approaches in the context of spatial relation learning.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Concept Of An Integrated Architecturementioning

confidence: 99%

Spatial relation learning in complementary scenarios with deep neural networks

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…Viewpoint planning (VPP) is an important part of various robotic applications to gain an understanding of the environment. For example, it enables a robot to identify occluded objects on a crowded table [1] or to estimate the amount of harvestable fruits in horticulture [2]. For manipulation purposes, VPP can also output the best grasping position by viewing and mapping the object from different angles [3].…”

Section: Introductionmentioning

confidence: 99%

Viewpoint Push Planning for Mapping of Unknown Confined Spaces

Dengler¹,

Pan²,

Kalagaturu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Viewpoint planning is an important task in any application where objects or scenes need to be viewed from different angles to achieve sufficient coverage. The mapping of confined spaces such as shelves is an especially challenging task since objects occlude each other and the scene can only be observed from the front, thus with limited possible viewpoints. In this paper, we propose a deep reinforcement learning framework that generates promising views aiming at reducing the map entropy. Additionally, the pipeline extends standard viewpoint planning by predicting adequate minimally invasive push actions to uncover occluded objects and increase the visible space. Using a 2.5D occupancy height map as state representation that can be efficiently updated, our system decides whether to plan a new viewpoint or perform a push. To learn feasible pushes, we use a neural network to sample push candidates on the map and have human experts manually label them to indicate whether the sampled push is a good action to perform. As simulated and real-world experimental results with a robotic arm show, our system is able to significantly increase the mapped space compared to different baselines, while the executed push actions highly benefit the viewpoint planner with only minor changes to the object configuration.

show abstract