Affordance graph: A framework to encode perspective taking and effort based affordances for day-to-day human-robot interaction

Abstract

Analyzing affordances has its root in sociocognitive development of primates. Knowing what the environment, including other agents, can offer in terms of action capabilities is important for our day-to-day interaction and cooperation. In this paper, we will merge two complementary aspects of affordances: from agent-object perspective, what an agent afford to do with an object, and from agent-agent perspective, what an agent can afford to do for other agent, and present a unified notion of Affordance Graph. The graph will encode affordances for a variety of tasks: take, give, pick, put on, put into, show, hide, make accessible, etc. Another novelty will be to incorporate the aspects of effort and perspectivetaking in constructing such graph. Hence, the Affordance Graph will tell about the action-capabilities of manipulating the objects among the agents and across the places, along with the information about the required level of efforts and the potential places. We will also demonstrate some interesting applications.

Building Affordance Relations for Robotic Agents - A Review

“…[Asada et al, 2009] summarises work that imitates the cognitive development of an infant, dividing it into 12 stages according to the difficulty of the motions. Inspired by [Asada et al, 2009] in this survey, we group actions in two sets: primitive actions, defined by simple motions, such as turning, moving forward, grasping or pushing [Abelha et al, 2016;Baleia et al, 2015;Seker et al, 2019], and compound actions defined by combining multiple simple motions, such as pouring or handing over an object [Price et al, 2016;Sun et al, 2014;.…”

“…The first general approach that we identify corresponds to methods that have full a priori task knowledge about the possible affordance relations. Work that uses this approach usually requires less complexity in their design choices [Sun et al, 2014;Moldovan and De Raedt, 2014;Byravan and Fox, 2017;Jiang et al, 2013;Koppula et al, 2016;Pieropan et al, 2014]. Namely, the information encapsulated in the affordance relation models of these methods represents their generalisation capabilities.…”

“…As a safety policy measure, the robot knows how to find a spatula that can help it reach the target object. In this case, there are no new relations learned [Pandey and Alami, 2013;Fallon et al, 2015;Diana et al, 2013;Marion et al, 2017]. As a result, the actions become rules that are queried at execution time.…”

“…Some approaches in this category attempt to perform multistep predictions based on known affordance relations, such as [Price et al, 2016;Cutsuridis and Taylor, 2013;Veres et al, 2020]. These methods often add a planning layer that allows them to achieve goal-oriented tasks by adapting to changes in the environment.…”

Expect the Unexpected: Leveraging the Human-Robot Ecosystem to Handle Unexpected Robot Failures

“…Develop robots that can imagine the world from another viewpoint, leveraging the perspectives views of users, bystanders, and other agents to clarify potential ambiguities, inform decision-making and resolve unexpected issues. Trafton et al (2005), Breazeal et al (2006), Warnier et al (2012, Pandey and Alami (2013), Pandey and Alami (2014), Milliez et al (2014), Milliez et al (2016), Fischer and Demiris (2016, Zhao et al (2016) Develop inter-firm networks and standardized protocols to ensure different robotic frameworks can communicate, cooperate and learn from one another. Mizukawa et al (2000), Bruyninckx (2001), Das et al (2005), Ferketic et al (2006, Schilling and Phelps (2007), Lechevalier et al (2011), Spulber (2013), Foucart and Li (2021 Enable and encourage units in the HRE to help one another.…”