To Pass or Not to Pass: Modeling the Movement and Affordance Dynamics of a Pick and Place Task

Abstract

Humans commonly engage in tasks that require or are made more efficient by coordinating with other humans. In this paper we introduce a task dynamics approach for modeling multi-agent interaction and decision making in a pick and place task where an agent must move an object from one location to another and decide whether to act alone or with a partner. Our aims were to identify and model (1) the affordance related dynamics that define an actor's choice to move an object alone or to pass it to their co-actor and (2) the trajectory dynamics of an actor's hand movements when moving to grasp, relocate, or pass the object. Using a virtual reality pick and place task, we demonstrate that both the decision to pass or not pass an object and the movement trajectories of the participants can be characterized in terms of a behavioral dynamics model. Simulations suggest that the proposed behavioral dynamics model exhibits features observed in human participants including hysteresis in decision making, non-straight line trajectories, and non-constant velocity profiles. The proposed model highlights how the same low-dimensional behavioral dynamics can operate to constrain multiple (and often nested) levels of human activity and suggests that knowledge of what, when, where and how to move or act during pick and place behavior may be defined by these low dimensional task dynamics and, thus, can emerge spontaneously and in real-time with little a priori planning.

A Hierarchical Behavioral Dynamic Approach for Naturally Adaptive Human‐Agent Pick‐and‐Place Interactions

“…With regard to pick-andplace behaviors in both individual and multiagent contexts, the hand movements of individuals engaged in a goal (target) directed reaching task can be modeled using a modified version of a behavioral dynamic model first introduced in the context of goal-directed locomotion [13,25,[35][36][37] (see also [9,11]). The model characterizes the movements of an agent's end-effector in terms of its heading direction, , such thaẗ…”

“…With regard to complex multiagent activity, behavioral dynamic models composed of dynamical motor primitives have been employed to capture and understand the stable patterns of coordinated perceptual-motor behavior across a range of discrete and rhythmic interpersonal task contexts, e.g., [15,[29][30][31]. Of particular relevance here is the recent work by Lamb and colleagues [13] demonstrating how collaborative pick-and-place behaviors could be modelled effectively using a hierarchical behavioral dynamic model that captured both the movement trajectories of actors and inter-actor pass decisions. That is, the behavioral dynamic model was able to effectively simulate the movement behaviors of the agent's end-effector (i.e., hand movements), as well as the dynamics of action selection (i.e., to pass or not to pass) during ongoing task behavior (for related research on modeling action selection dynamics see, e.g., [32,33]).…”

“…That is, the behavioral dynamic model was able to effectively simulate the movement behaviors of the agent's end-effector (i.e., hand movements), as well as the dynamics of action selection (i.e., to pass or not to pass) during ongoing task behavior (for related research on modeling action selection dynamics see, e.g., [32,33]). The aim of the current study is to extend this latter work and bridge the work on dynamical movement primitives and behavioral dynamic modeling, by investigating whether multiagent hierarchical behavioral dynamic models derived from dynamical movement primitives can employed to control artificial agents for HMI/HRI [4,13,30,31,34]. Indeed, although dynamic movement primitive models have been implemented in HRI contexts, they are not typically derived from human movement data.…”

“…Building on data observations from a human-human interaction (HHI) experiment exploring the behavioral dynamics of a more interactive (complicated) pick-and-place task than was employed by Lamb et al [13], in which only one agent made decisions and initiated task actions. In the current experimental task, both coactors could make decisions about and initiate task actions, allowing for the possibility of greater influence and range of task action combinations.…”

“…In the current experimental task, both coactors could make decisions about and initiate task actions, allowing for the possibility of greater influence and range of task action combinations. Further, we implemented an extended version of the hierarchical behavioral dynamic model proposed by Lamb et al [13] within the control architecture of an artificial human agent in virtual reality. The proposed pick-and-place agent (PAPA) was expected to illustrate how this hierarchical behavioral dynamic model can be used not only to capture and predict human behavior, but also to enact human behaviors in a collaborative human-agent interaction (HAI) task context as a generalized form of HRI/HMI.…”