Online Identification of Interaction Behaviors From Haptic Data During Collaborative Object Transfer

Abstract

Joint object transfer is a complex task, which is less structured and less specific than what is existing in several industrial settings. When two humans are involved in such a task, they cooperate through different modalities to understand the interaction states during operation and mutually adapt to one another's actions. Mutual adaptation implies that both partners can identify how well they collaborate (i.e. infer about the interaction state) and act accordingly. These interaction states can define whether the partners work in harmony, face conflicts, or remain passive during interaction. Understanding how two humans work together during physical interactions is important when exploring the ways a robotic assistant should operate under similar settings. This study acts as a first step to implement an automatic classification mechanism during ongoing collaboration to identify the interaction state during object comanipulation. The classification is done on a dataset consisting of data from 40 subjects, who are partnered to form 20 dyads. The dyads experiment in a physical human-human interaction (pHHI) scenario to move an object in an haptics-enabled virtual environment to reach predefined goal configurations. In this study, we propose a sliding-window approach for feature extraction and demonstrate the online classification methodology to identify interaction patterns. We evaluate our approach using 1) a support vector machine classifier (SVMc) and 2) a Gaussian Process classifier (GPc) for multi-class classification, and achieve over 80% accuracy with both classifiers when identifying general interaction types.

A Novel Haptic Feature Set for the Classification of Interactive Motor Behaviors in Collaborative Object Transfer

“…Using a random forest classifier, we verified that haptic information, i.e., wrenches, was sufficient to differentiate between the interactions patterns. This contrasts with our earlier work [25], [26], which suggested that kinematic information is needed to improve classification rates in distinguishing between interaction patterns in dyadic co-manipulation. In this work, we have demonstrated that using carefully devised haptic-related metrics derived from raw data eliminates the dependency on kinematics.…”

“…In future work, we will use the proposed features with online classification as proposed in [26], and develop a system that identify the current interaction state in real-time during collaboration. Although this experiment does not cover all the human-human interaction patterns, this information will be useful when programming a robot partner that reads interaction states and reactively acts in response to the predicted behaviors, which is the ultimate goal of this research.…”

“…In [25], we investigated the use of raw interaction data, and extracted features by processing the partners' applied forces and the manipulated object's velocity from annotated data. Later, we proposed an online feature extraction method and demonstrated its use for identifying interaction patterns in real-time collaboration using similar features [26]. In both studies, we observed that raw haptic data were insufficient to achieve good classification rates and the object pose information was needed.…”

Intent-Aware Predictive Haptic Guidance and its Application to Shared Control Teleoperation

“…To tackle this, active learning mechanisms can be integrated in the current framework to enable iterative learning to enrich the robot's behaviour library. In addition, extending on our recent work [36], [37], we plan to integrate active conflict recognition within this framework, and as a result, handle not only collaborative but also conflicting scenarios in physical human-robot interaction.…”

Modulation of Control Authority in Adaptive HapticShared Control Paradigms

“…Therefore, it is essential to develop an approach to estimate the parameters of human biomechanics and track them in real-time as they vary. Furthermore, it is crucial to create a method that allows recognizing the current interaction mode in real-time using the data acquired by on-board sensors [38]. By knowing the interaction mode, an appropriate cost function can be defined, and the automation system can adjust its behavior based on this cost function.…”