Attentional supervision of human-robot collaborative plans

Abstract

In human-robot interactive scenarios communication and collaboration during task execution are crucial issues. Since the human behavior is unpredictable and ambiguous, an interactive robotic system is to continuously interpret intentions and goals adapting its executive and communicative processes according to the users behaviors. In this work, we propose an integrated system that exploits attentional mechanisms to flexibly adapt planning and executive processes to the multimodal human-robot interaction.

A Robotic Cognitive Control Framework for Collaborative Task Execution and Learning

“…In Caccavale et al. (2016), we also show how multiple alternative methods competing in WM for the same task can support fast opportunistic plan repairs when the human behavior diverges from the planned one.…”

“…Therefore, in contrast to typical AI plan-monitoring systems, the allocated plans do not fully constraint the execution, instead they provide attentional guidance used to bias the executive system toward the accomplishment of planned activities. In Caccavale et al (2016), we also show how multiple alternative methods competing in WM for the same task can support fast opportunistic plan repairs when the human behavior diverges from the planned one.…”

“…We are particularly interested in interaction scenarios where executive attention is affected by multimodal sources (e.g., utterance, gaze direction, gestures, physical interaction, body postures, etc.). Preliminary examples of integrated frameworks for multimodal attention-based interaction can be found in Caccavale et al (2016). We are also investigating the effectiveness of the described attention-based executive framework for long-term autonomy in complex everyday scenarios.…”

“…More complex activity recruiting mechanisms as in Anderson, Matessa, and Lebiere (1997) and Franklin, Madl, and D'Mello (2014) may be introduced as well. Since the LTM schemata can be mapped into HTN planning domains (Caccavale, Cacace, Fiore, Alami, & Finzi, 2016), a task hierarchy instance can also be externally generated by an HTN planner (de Silva, Lallement, & Alami, 2015; Nau et al., 2003) and then directly allocated in WM. Multiple plans may also be allocated in WM, while concrete activity execution depends on the attention regulations and contention‐scheduling mechanisms.…”

A Hierarchical Architecture for Human-Robot Cooperation Processes

“…Such a layer is expected to enforce effectiveness since it constitutes a close-to-human language used to associate semantics to each robot's action [31]. A cognitive approach based on an attention-based mechanism is proposed by [32], in which plans are generated using hierarchical task networks, and an attention-based system executes and monitors multiple plans while resolving possible conflicts. To demonstrate capabilities of the approach, a pick and place task in a simulated environment is shown, therefore eliminating intrinsic uncertainties characterising perception and action in real-world environments.…”