Bimanual telemanipulation with force and haptic feedback through an anthropomorphic avatar system

Lenz, Christian; Behnke, Sven

doi:10.1016/j.robot.2022.104338

Cited by 21 publications

(6 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Physical avatars are often implemented with robots capable of locomotion. Typical solutions include multi-legged or wheeled robots (7)(8)(9)(10). In contexts where remote interaction with humans is crucial, humanoid robot avatars show great potential for existing and future applications.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

iCub3 avatar system: Enabling remote fully immersive embodiment of humanoid robots

Dafarra,

Pattacini,

Romualdi

et al. 2024

Sci. Robot.

View full text Add to dashboard Cite

We present an avatar system designed to facilitate the embodiment of humanoid robots by human operators, validated through iCub3, a humanoid developed at the Istituto Italiano di Tecnologia. More precisely, the paper makes two contributions: First, we present the humanoid iCub3 as a robotic avatar that integrates the latest significant improvements after about 15 years of development of the iCub series. Second, we present a versatile avatar system enabling humans to embody humanoid robots encompassing aspects such as locomotion, manipulation, voice, and facial expressions with comprehensive sensory feedback including visual, auditory, haptic, weight, and touch modalities. We validated the system by implementing several avatar architecture instances, each tailored to specific requirements. First, we evaluated the optimized architecture for verbal, nonverbal, and physical interactions with a remote recipient. This testing involved the operator in Genoa and the avatar in the Biennale di Venezia, Venice—about 290 kilometers away—thus allowing the operator to visit the Italian art exhibition remotely. Second, we evaluated the optimized architecture for recipient physical collaboration and public engagement on stage, live, at the We Make Future show, a prominent world digital innovation festival. In this instance, the operator was situated in Genoa while the avatar operated in Rimini—about 300 kilometers away—interacting with a recipient who entrusted the avatar with a payload to carry on stage before an audience of approximately 2000 spectators. Third, we present the architecture implemented by the iCub Team for the All Nippon Airways (ANA) Avatar XPrize competition.

show abstract

Section: Introductionmentioning

confidence: 99%

“…The devices are often VR commercial products (23)(24)(25) or motion capture systems (13,14). To achieve bilateral feedback teleoperation, it is possible to leverage ad hoc designed exoskeletons (8,26). However, exoskeletons can be very invasive, thus constraining the operator's motion in sometimes narrow envelopes.…”

Section: Introductionmentioning

confidence: 99%

iCub3 avatar system: Enabling remote fully immersive embodiment of humanoid robots

Dafarra,

Pattacini,

Romualdi

et al. 2024

Sci. Robot.

View full text Add to dashboard Cite

show abstract

“…These studies indicate the importance of agents’ reactive behaviors to human touch in conveying human-like impressions to remote users. In avatar systems, the haptic sensations are often utilized to control avatars intuitively and to understand remote environments ( Salisbury, Conti & Barbagli, 2004 ; Colella et al, 2019 ; Ho & Nakayama, 2021 ; Tanaka et al, 2022 ; Lenz & Behnke, 2023 ). These studies mainly focus on reproducing realistic haptic sensations for the avatar’s operator.…”

Section: Introductionmentioning

confidence: 99%

“…The haptic sensation is fundamental feedback to humans for understanding and interacting with their surroundings ( Robles-De-La-Torre, 2006 ). To achieve immersive and intuitive control of avatars, various devices and mechanisms have been proposed to communicate haptic sensations to people in different locations ( Salisbury, Conti & Barbagli, 2004 ; Colella et al, 2019 ; Ho & Nakayama, 2021 ; Tanaka et al, 2022 ; Lenz & Behnke, 2023 ). For example, Colella et al (2019) proposed a device that offers proprioceptive information of the opening of an artificial hand through sensory substitution, specifically by longitudinal skin stretch stimulation.…”

Section: Introductionmentioning

confidence: 99%

“…For example, Colella et al (2019) proposed a device that offers proprioceptive information of the opening of an artificial hand through sensory substitution, specifically by longitudinal skin stretch stimulation. Lenz & Behnke (2023) introduced a bimanual telemanipulation system that blends an anthropomorphic avatar robot with an operator station that delivers force and haptic feedback for enhancing immersion and task efficiency in robotic teleoperation.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The experience of a tele-operated avatar being touched increases operator’s sense of discomfort

Kimoto,

Shiomi

2024

PeerJ Computer Science

View full text Add to dashboard Cite

Recent advancements in tele-operated avatars, both on-screen and robotic, have expanded opportunities for human interaction that exceed spatial and physical limitations. While numerous studies have enhanced operator control and improved the impression left on remote users, one area remains underexplored: the experience of operators during touch interactions between an avatar and a remote interlocutor. Touch interactions have become commonplace with avatars, especially those displayed on or integrated with touchscreen interfaces. Although the need for avatars to exhibit human-like touch responses has been recognized as beneficial for maintaining positive impressions on remote users, the sensations and experiences of the operators behind these avatars during such interactions remain largely uninvestigated. This study examines the sensations felt by an operator when their tele-operated avatar is touched remotely. Our findings reveal that operators can perceive a sensation of discomfort when their on-screen avatar is touched. This feeling is intensified when the touch is visualized and the avatar reacts to it. Although these autonomous responses may enhance the human-like perceptions of remote users, they might also lead to operator discomfort. This situation underscores the importance of designing avatars that address the experiences of both remote users and operators. We address this issue by proposing a tele-operated avatar system that minimizes unwarranted touch interactions from unfamiliar interlocutors based on social intimacy.

show abstract

Evaluation of Tactile Feedback for Teleoperated Glove-Based Interaction Tasks

Dural,

Shahid,

Gioioso

et al. 2024

Springer Proceedings in Advanced Robotics

View full text Add to dashboard Cite

Bimanual telemanipulation with force and haptic feedback through an anthropomorphic avatar system

Cited by 21 publications

References 35 publications

iCub3 avatar system: Enabling remote fully immersive embodiment of humanoid robots

iCub3 avatar system: Enabling remote fully immersive embodiment of humanoid robots

The experience of a tele-operated avatar being touched increases operator’s sense of discomfort

Evaluation of Tactile Feedback for Teleoperated Glove-Based Interaction Tasks

Contact Info

Product

Resources

About