An Effective Head-Based HRI for 6D Robotic Grasping Using Mixed Reality

Zhang, Chengjie; Leng, Yuquan; Fu, Zezheng; Yao-yu, Cheng; Fu, Chenglong

doi:10.1109/lra.2023.3261701

Cited by 7 publications

(2 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…et al presents the use of a virtual reality digital twin of a physical layout as a mechanism to understand human reactions to both predictable and unpredictable robot motions. A set of established metrics as well as a newly developed kinetic energy ratio metric is used to analyse human reactions and validate the effectiveness of the virtual reality environment [95]. Duguleana, M. et al present an analysis of virtual reality (VR) as an alternative to real-world applications for testing material-handling scenarios that involve collaboration between robots and humans.…”

Section: Virtual Reality (Vr)-based Cobotmentioning

confidence: 99%

COBOT Applications—Recent Advances and Challenges

2023

View full text Add to dashboard Cite

This study provides a structured literature review of the recent COllaborative roBOT (COBOT) applications in industrial and service contexts. Several papers and research studies were selected and analyzed, observing the collaborative robot interactions, the control technologies and the market impact. This review focuses on stationary COBOTs that may guarantee flexible applications, resource efficiency, and worker safety from a fixed location. COBOTs offer new opportunities to develop and integrate control techniques, environmental recognition of time-variant object location, and user-friendly programming to interact safely with humans. Artificial Intelligence (AI) and machine learning systems enable and boost the COBOT’s ability to perceive its surroundings. A deep analysis of different applications of COBOTs and their properties, from industrial assembly, material handling, service personal assistance, security and inspection, Medicare, and supernumerary tasks, was carried out. Among the observations, the analysis outlined the importance and the dependencies of the control interfaces, the intention recognition, the programming techniques, and virtual reality solutions. A market analysis of 195 models was developed, focusing on the physical characteristics and key features to demonstrate the relevance and growing interest in this field, highlighting the potential of COBOT adoption based on (i) degrees of freedom, (ii) reach and payload, (iii) accuracy, and (iv) energy consumption vs. tool center point velocity. Finally, a discussion on the advantages and limits is summarized, considering anthropomorphic robot applications for further investigations.

show abstract

Section: Virtual Reality (Vr)-based Cobotmentioning

confidence: 99%

COBOT Applications—Recent Advances and Challenges

2023

View full text Add to dashboard Cite

show abstract

“…This research encompasses various applications of WRLs, including affixing them to the shoulders to assist with overhead support tasks [4], wearing them around the waist to aid in daily life activities [5], mounting them on the back for carrying loads [6], and incorporating supplementary robotic digits to enhance grip strength [7] and piano playing skills [8]. Additionally, there have been studies focusing on improving the quality of life for individuals with disabilities, such as using extra robotic limbs to aid hemiplegic patients during eating [9], developing robotic arms capable of object grasping based on voice and head movements [10], and utilizing extra robotic fingers to assist individuals with disabilities in grasping objects [11]. In these applications, the primary function of WRLs is to collaborate with human users and facilitate tasks such as providing support, grasping objects, and manipulation.…”

Section: Introductionmentioning

confidence: 99%

A Mouth and Tongue Interactive Device to Control Wearable Robotic Limbs in Tasks where Human Limbs Are Occupied

Jing,

Zheng,

Zhang

et al. 2024

Biosensors

View full text Add to dashboard Cite

The Wearable Robotic Limb (WRL) is a type of robotic arm worn on the human body, aiming to enhance the wearer’s operational capabilities. However, proposing additional methods to control and perceive the WRL when human limbs are heavily occupied with primary tasks presents a challenge. Existing interactive methods, such as voice, gaze, and electromyography (EMG), have limitations in control precision and convenience. To address this, we have developed an interactive device that utilizes the mouth and tongue. This device is lightweight and compact, allowing wearers to achieve continuous motion and contact force control of the WRL. By using a tongue controller and mouth gas pressure sensor, wearers can control the WRL while also receiving sensitive contact feedback through changes in mouth pressure. To facilitate bidirectional interaction between the wearer and the WRL, we have devised an algorithm that divides WRL control into motion and force-position hybrid modes. In order to evaluate the performance of the device, we conducted an experiment with ten participants tasked with completing a pin-hole assembly task with the assistance of the WRL system. The results show that the device enables continuous control of the position and contact force of the WRL, with users perceiving feedback through mouth airflow resistance. However, the experiment also revealed some shortcomings of the device, including user fatigue and its impact on breathing. After experimental investigation, it was observed that fatigue levels can decrease with training. Experimental studies have revealed that fatigue levels can decrease with training. Furthermore, the limitations of the device have shown potential for improvement through structural enhancements. Overall, our mouth and tongue interactive device shows promising potential in controlling the WRL during tasks where human limbs are occupied.

show abstract