Collision-free human-robot collaboration based on context awareness

Liu, Hongyi; Wang, Lihui

doi:10.1016/j.rcim.2020.101997

Cited by 99 publications

(49 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Because AR/MR can embed 3D virtual information onto the real environment, they are widely used in various fields such as HRI, manufacturing, and robot manipulation [2], [5], [6], [9], [38]. Huy et al [37] proposed a new interface framework for HRI using a laser-writer instead of a projector -suitable for indoor and outdoor applications.…”

Section: Hri Applications Using Ar/mr and Deep Learningmentioning

confidence: 99%

“…Because existing industrial robots are both programmed and assigned to perform repetitive tasks, it is difficult to cope with uncertainties when an unexpected situation occurs or the work environment changes. However, collaborative robots can perform intelligent tasks even in dynamic and uncertain situations by utilizing various sensors, such as RGB-D sensors and pressure sensors, enabling safe collaboration between the human operator and the robot by preventing collisions between them [1], [2]. Typically, a new interaction method and an interface tool to manipulate the robot are required for supporting effective HRI, as the operator mainly interacts with the robot through a 2D interface such as a teach pendant or a keyboard mouse.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Hands-Free Human–Robot Interaction Using Multimodal Gestures and Deep Learning in Wearable Mixed Reality

et al. 2021

View full text Add to dashboard Cite

This study proposes a novel hands-free interaction method using multimodal gestures such as eye gazing and head gestures and deep learning for human-robot interaction (HRI) in mixed reality (MR) environments. Since human operators hold some objects for conducting tasks, there are many constrained situations where they cannot use their hands for HRI interactions. To provide more effective and intuitive task assistance, the proposed hands-free method supports coarse-to-fine interactions. Eye gazing-based interaction is used for coarse interactions such as searching and previewing of target objects, and head gesture interactions are used for fine interactions such as selection and 3D manipulation. In addition, deep learning-based object detection is applied to estimate the initial positioning of physical objects to be manipulated by the robot. The result of object detection is then combined with 3D spatial mapping in the MR environment for supporting accurate initial object positioning. Furthermore, virtual object-based indirect manipulation is proposed to support more intuitive and efficient control of the robot, compared with traditional direct manipulation (e.g., joint-based and end effector-based manipulations). In particular, a digital twin, the synchronized virtual robot of the real robot, is used to provide a preview and simulation of the real robot to manipulate it more effectively and accurately. Two case studies were conducted to confirm the originality and advantages of the proposed hands-free HRI: (1) performance evaluation of initial object positioning and (2) comparative analysis with traditional direct robot manipulations. The deep learningbased initial positioning reduces much effort for robot manipulation using eye gazing and head gestures. The object-based indirect manipulation also supports more effective HRI than previous direct interaction methods. INDEX TERMS Deep learning, eye gazing, hands-free interaction, head gestures, human-robot interaction, mixed reality, object detection.

show abstract

Section: Hri Applications Using Ar/mr and Deep Learningmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Hands-Free Human–Robot Interaction Using Multimodal Gestures and Deep Learning in Wearable Mixed Reality

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Keeping a predefined distance between the robot and the human is a safety measurement that will, in all motions, interfere with the robot's path planning. Different types of sensors and several strategies have been adopted to avoid potential collisions by jointly considering aspects of human monitoring and motion planning [44][45][46][47][48][49][50][51][52][53][54][55][56][57][58]. When it comes to physical humanrobot interaction (pHRI), ref.…”

Section: Safety In Hrcmentioning

confidence: 99%

“…The main challenges of reinforcement learning based robotics are discussed in the review paper by Kober et al [150], including different solutions to address them. Reinforcement learning and imitation learning, combined with deep learning techniques, start offering novel computational tools for robotic skill acquisition and control problems, such as robotic manipulation [56,[151][152][153] and robot grasping [101,154]. These methods are attracting much attention since the robot can automatically learn skills from the sensory inputs with minimal engineering.…”

Section: Limitations and Opportunities For Cognitive Collaborationmentioning

confidence: 99%

Trends of Human-Robot Collaboration in Industry Contexts: Handover, Learning, and Metrics

Castro

Silva

Santos

2021

Sensors

View full text Add to dashboard Cite

Repetitive industrial tasks can be easily performed by traditional robotic systems. However, many other works require cognitive knowledge that only humans can provide. Human-Robot Collaboration (HRC) emerges as an ideal concept of co-working between a human operator and a robot, representing one of the most significant subjects for human-life improvement.The ultimate goal is to achieve physical interaction, where handing over an object plays a crucial role for an effective task accomplishment. Considerable research work had been developed in this particular field in recent years, where several solutions were already proposed. Nonetheless, some particular issues regarding Human-Robot Collaboration still hold an open path to truly important research improvements. This paper provides a literature overview, defining the HRC concept, enumerating the distinct human-robot communication channels, and discussing the physical interaction that this collaboration entails. Moreover, future challenges for a natural and intuitive collaboration are exposed: the machine must behave like a human especially in the pre-grasping/grasping phases and the handover procedure should be fluent and bidirectional, for an articulated function development. These are the focus of the near future investigation aiming to shed light on the complex combination of predictive and reactive control mechanisms promoting coordination and understanding. Following recent progress in artificial intelligence, learning exploration stand as the key element to allow the generation of coordinated actions and their shaping by experience.

show abstract

“…This brings important benefits for navigating inside a factory in everyday conditions since we are very interested in robustness and accuracy. Besides, other works focus not only on collision avoidance for safety, but also on maintaining the efficiency of manufacturing processes [28]; while focused on human-robot collaboration, a similar approach could be explored in case more than one aerial robot is used in a factory. The adopted approach should have a very low computational cost since all the modules of the proposed robotic architecture are executed onboard.…”

Section: Related Workmentioning

confidence: 99%

Introducing autonomous aerial robots in industrial manufacturing

Pérez-Grau

Dios

Paneque

et al. 2021

Journal of Manufacturing Systems

View full text Add to dashboard Cite

Although ground robots have been successfully used for many years in manufacturing, the capability of aerial robots to agilely navigate in the often sparse and static upper part of factories makes them suitable for performing tasks of interest in many industrial sectors. This paper presents the design, development, and validation of a fully autonomous aerial robotic system for manufacturing industries. It includes modules for accurate pose estimation without using a Global Navigation Satellite System (GNSS), autonomous navigation, radio-based localization, and obstacle avoidance, among others, providing a fully onboard solution capable of autonomously performing complex tasks in dynamic indoor environments in which all necessary sensors, electronics, and processing are on the robot. It was developed to fulfill two use cases relevant in many industries: light object logistics and missing tool search. The presented robotic system, functionalities, and use cases have been extensively validated with Technology Readiness Level 7 (TRL-7) in the Centro Bahía de Cádiz (CBC) Airbus D&S factory in fully working conditions.

show abstract

Collision-free human-robot collaboration based on context awareness

Cited by 99 publications

References 29 publications

Hands-Free Human–Robot Interaction Using Multimodal Gestures and Deep Learning in Wearable Mixed Reality

Hands-Free Human–Robot Interaction Using Multimodal Gestures and Deep Learning in Wearable Mixed Reality

Trends of Human-Robot Collaboration in Industry Contexts: Handover, Learning, and Metrics

Introducing autonomous aerial robots in industrial manufacturing

Contact Info

Product

Resources

About