Geometric Affordance Perception: Leveraging Deep 3D Saliency With the Interaction Tensor

Ochoa, Eguzkine; Mayol-Cuevas, Walterio

doi:10.3389/fnbot.2020.00045

Cited by 12 publications

(6 citation statements)

References 74 publications

(95 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In further studies, Ruiz and Mayol-Cuevas (2020) developed a geometric interaction descriptor for non-articulated, rigid object shapes. Given a 3D environment, the method demonstrated good generalization on detecting physically feasible object–environment configurations.…”

Section: Related Workmentioning

confidence: 99%

“…We are inspired by recent methods that have revisited geometric features, such as the bisector surface for scene–object indexing ( Zhao et al., 2014 ) and affordance detection ( Ruiz and Mayol-Cuevas, 2020 ). Initiating from a spatial representation makes sense if it helps reduce data training needs and simplify explanations—as long as it can outperform data-intensive approaches.…”

Section: Arosmentioning

confidence: 99%

“…Given two surfaces

, the BS is the set of sphere centers that touch both surfaces at one point each. Due to its stability and geometrical characteristics, the IBS has been used in context retrieval, interaction classification, and functionality analysis ( Zhao et al., 2014 ; Hu et al., 2015 ; Hu et al., 2016 ; Zhao et al., 2016 ; Zhao et al., 2017 ; Ruiz and Mayol-Cuevas, 2020 ). Our approach expands on these ideas and is geometrically intuitive and straightforward.…”

Section: Arosmentioning

confidence: 99%

See 2 more Smart Citations

AROS: Affordance Recognition with One-Shot Human Stances

Pacheco-Ortega

Mayol-Cuevas

2023

Front. Robot. AI

Self Cite

View full text Add to dashboard Cite

We present Affordance Recognition with One-Shot Human Stances (AROS), a one-shot learning approach that uses an explicit representation of interactions between highly articulated human poses and 3D scenes. The approach is one-shot since it does not require iterative training or retraining to add new affordance instances. Furthermore, only one or a small handful of examples of the target pose are needed to describe the interactions. Given a 3D mesh of a previously unseen scene, we can predict affordance locations that support the interactions and generate corresponding articulated 3D human bodies around them. We evaluate the performance of our approach on three public datasets of scanned real environments with varied degrees of noise. Through rigorous statistical analysis of crowdsourced evaluations, our results show that our one-shot approach is preferred up to 80% of the time over data-intensive baselines.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Arosmentioning

confidence: 99%

“…Given two surfaces

Section: Arosmentioning

confidence: 99%

See 1 more Smart Citation

AROS: Affordance Recognition with One-Shot Human Stances

Pacheco-Ortega

Mayol-Cuevas

2023

Front. Robot. AI

Self Cite

View full text Add to dashboard Cite

show abstract

“…[9], [10], [14], [16], [17] use convolutional neural networks (CNNs) to detect regions of affordance in an image. Ruiz and Mayol-Cuevas [11] predict affordance candidate locations in environments via the interaction tensor. In contrast to detecting different types of affordances, we focus on understanding the sitting affordance and use it for real-robot experiment.…”

Section: Related Workmentioning

confidence: 99%

Put the Bear on the Chair! Intelligent Robot Interaction with Previously Unseen Chairs via Robot Imagination

Wu¹,

Meng²,

Ruan³

et al. 2021

Preprint

View full text Add to dashboard Cite

In this letter, we study the problem of autonomously placing a teddy bear on a previously unseen chair for sitting. To achieve this goal, we present a novel method for robots to imagine the sitting pose of the bear by physically simulating a virtual humanoid agent sitting on the chair. We also develop a robotic system which leverages motion planning to plan SE(2) motions for a humanoid robot to walk to the chair and whole-body motions to put the bear on it, respectively. Furthermore, to cope with the cases where the chair is not in an accessible pose for placing the bear, a human-robot interaction (HRI) framework is introduced in which a human follows language instructions given by the robot to rotate the chair and help make the chair accessible. We implement our method with a robot arm and a humanoid robot. We calibrate the proposed system with 3 chairs and test on 12 previously unseen chairs in both accessible and inaccessible poses extensively. Results show that our method enables the robot to autonomously put the teddy bear on the 12 unseen chairs with a very high success rate. The HRI framework is also shown to be very effective in changing the accessibility of the chair. Source code will be available. Video demos are available at https: //chirikjianlab.github.io/putbearonchair/.

show abstract

“…Note that, our distance-based interaction representation explicitly captures the contact and the proximal relationships between the human body and the scene and can be regarded as the scene 'affordance'. Compared with other person-scene interaction representations [7,30,31,32,34], ours is purely geometry-based, is more efficient than dense affordance maps, and requires neither scene semantics nor action type labeling, nor affordance class annotation.…”

Section: Introductionmentioning

confidence: 99%