A New Mechanism for Collision Detection in Human–Robot Collaboration using Deep Learning Techniques

Rodrigues, Iago Richard; Barbosa, Gibson; Filho, Assis Oliveira; Cani, Carolina; Souza, Ricardo M. Campello de; Kelner, Judith; Souza, Ricardo; Marquezini, M. V.; Lins, Silvia

doi:10.1007/s40313-021-00829-3

Cited by 5 publications

(3 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yet, these models were not deployed for human-robot task completion. Further developments include the work by Rodrigues et al [32], which presents a CNN model for detecting collisions between users and cobots with an accuracy of 89%, and by Morrison et al [51], which develops a generative grasping CNNbased model achieving between 88% and 94% accuracy for grasping static and moving objects. Additionally, Sekkat et al [29] utilized a reinforcement learning model for grasping tasks with a 5-DOF robot, reporting 95.5% accuracy, albeit without human interaction.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Integration of Deep Learning and Collaborative Robot for Assembly Tasks

Mendez,

Ochoa,

Olivera-Guzman

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

Human–robot collaboration has gained attention in the field of manufacturing and assembly tasks, necessitating the development of adaptable and user-friendly forms of interaction. To address this demand, collaborative robots (cobots) have emerged as a viable solution. Deep Learning has played a pivotal role in enhancing robot capabilities and facilitating their perception and understanding of the environment. This study proposes the integration of cobots and Deep Learning to assist users in assembly tasks such as part handover and storage. The proposed system includes an object classification system to categorize and store assembly elements, a voice recognition system to classify user commands, and a hand-tracking system for close interaction. Tests were conducted for each isolated system and for the complete application as used by different individuals, yielding an average accuracy of 91.25%. The integration of Deep Learning into cobot applications has significant potential for transforming industries, including manufacturing, healthcare, and assistive technologies. This work serves as a proof of concept for the use of several neural networks and a cobot in a collaborative task, demonstrating communication between the systems and proposing an evaluation approach for individual and integrated systems.

show abstract

Section: Discussionmentioning

confidence: 99%

“…This capability has been utilized for route control and pick-and-place routines [29,30]. Additionally, computer vision in cobots has been employed for safety purposes, including gesture recognition and human position tracking [31,32], where vision systems integrated with neural networks are applied to track the body of the user.…”

Section: Introductionmentioning

confidence: 99%

Integration of Deep Learning and Collaborative Robot for Assembly Tasks

Mendez,

Ochoa,

Olivera-Guzman

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…The specific standard provides a series of safety guidelines depending on the level of interaction and can be used complementarily to other ISO guidelines associated with robotic processes, such as ISO 10218-1:2011 [ 17 ], ISO 10218-2:2011 [ 18 ], and ISO 13855 [ 19 , 20 ]. Yet, it should be noted that the maturity and readiness of vision systems in industrial environments are still under review, since in some specific scenarios the detection of an operator may be prevented due to occlusions [ 21 , 22 ]. In these cases, using various types of sensors in conjunction with sensor fusion algorithms has been reported as a method to improve the overall perception of the process of human pose estimation for collaborative robotic applications [ 23 ].…”

Section: Introductionmentioning

confidence: 99%

On the Evaluation of Diverse Vision Systems towards Detecting Human Pose in Collaborative Robot Applications

Ramasubramanian,

Kazasidis,

Fay

et al. 2024

Sensors

View full text Add to dashboard Cite

Tracking human operators working in the vicinity of collaborative robots can improve the design of safety architecture, ergonomics, and the execution of assembly tasks in a human–robot collaboration scenario. Three commercial spatial computation kits were used along with their Software Development Kits that provide various real-time functionalities to track human poses. The paper explored the possibility of combining the capabilities of different hardware systems and software frameworks that may lead to better performance and accuracy in detecting the human pose in collaborative robotic applications. This study assessed their performance in two different human poses at six depth levels, comparing the raw data and noise-reducing filtered data. In addition, a laser measurement device was employed as a ground truth indicator, together with the average Root Mean Square Error as an error metric. The obtained results were analysed and compared in terms of positional accuracy and repeatability, indicating the dependence of the sensors’ performance on the tracking distance. A Kalman-based filter was applied to fuse the human skeleton data and then to reconstruct the operator’s poses considering their performance in different distance zones. The results indicated that at a distance less than 3 m, Microsoft Azure Kinect demonstrated better tracking performance, followed by Intel RealSense D455 and Stereolabs ZED2, while at ranges higher than 3 m, ZED2 had superior tracking performance.

show abstract