Pose estimation and robotic insertion tasks based on YOLO and layout features

Mou, Fangli; Ren, Hao; Wang, Bin; Wu, Dan

doi:10.1016/j.engappai.2022.105164

Cited by 9 publications

(3 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The relative pose of the master object in the robot's base frame

can be obtained from extrinsic and intrinsic camera parameters by hand-eye calibration and YOLO-based detectors fine-tuned to the domain. The YOLO algorithm is widely used to detect objects in the image or video streams (Mou et al, 2022 ). For each object in the image I eth , the algorithm makes multiple predictions of bounding boxes that contain information concerning the object's position ( x, y ), size ( w, h ), confidence c con , and category c cate , as shown in Equation (2) .…”

Section: Methodsmentioning

confidence: 99%

Extended residual learning with one-shot imitation learning for robotic assembly in semi-structured environment

Wang,

Su,

Sun

et al. 2024

Front. Neurorobot.

View full text Add to dashboard Cite

IntroductionRobotic assembly tasks require precise manipulation and coordination, often necessitating advanced learning techniques to achieve efficient and effective performance. While residual reinforcement learning with a base policy has shown promise in this domain, existing base policy approaches often rely on hand-designed full-state features and policies or extensive demonstrations, limiting their applicability in semi-structured environments.MethodsIn this study, we propose an innovative Object-Embodiment-Centric Imitation and Residual Reinforcement Learning (OEC-IRRL) approach that leverages an object-embodiment-centric (OEC) task representation to integrate vision models with imitation and residual learning. By utilizing a single demonstration and minimizing interactions with the environment, our method aims to enhance learning efficiency and effectiveness. The proposed method involves three key steps: creating an object-embodiment-centric task representation, employing imitation learning for a base policy using via-point movement primitives for generalization to different settings, and utilizing residual RL for uncertainty-aware policy refinement during the assembly phase.ResultsThrough a series of comprehensive experiments, we investigate the impact of the OEC task representation on base and residual policy learning and demonstrate the effectiveness of the method in semi-structured environments. Our results indicate that the approach, requiring only a single demonstration and less than 1.2 h of interaction, improves success rates by 46% and reduces assembly time by 25%.DiscussionThis research presents a promising avenue for robotic assembly tasks, providing a viable solution without the need for specialized expertise or custom fixtures.

show abstract

“…The relative pose of the master object in the robot's base frame

Section: Methodsmentioning

confidence: 99%

Extended residual learning with one-shot imitation learning for robotic assembly in semi-structured environment

Wang,

Su,

Sun

et al. 2024

Front. Neurorobot.

View full text Add to dashboard Cite

show abstract

“…Similarly, a vision-based grasping system was proposed in 30 , where the neural network uses the point cloud of the grasp to classify the quality of the grasping, utilizing 3D scans and RGB-D pictures of known objects. Work in 31 proposed a novel grasping framework for sorting bottles in a complex environment, exploiting the Region Proposal Network (RPN) for object recognition and pose estimation 32 . Moreover, an intelligent grasping approach was proposed in 33 for picking objects from the floor using the method of highly accumulated features (HAF).…”

Section: Standard Vision-based Robotic Grasping Systemmentioning

confidence: 99%

Efficient Event-Based Robotic Grasping Perception using Hyperdimensional Computing

Hassan,

Zou,

Chen

et al. 2023

Preprint

View full text Add to dashboard Cite

Grasping is Fundamental in various robotic applications, particularly within industrial contexts. Accurate inference of object properties is a crucial step toward enhancing grasping quality. Dynamic and Active Vision Sensors (DAVIS), increasingly utilized for robotic grasping, offer superior energy efficiency, lower latency, and higher temporal resolution than traditional cameras. However, the data they generate can be complex and noisy, necessitating substantial pre-processing. In response to these challenges, we introduce GraspHD, an innovative end-to-end algorithm that leverages brain-inspired hyperdimensional computing (HDC) to learn about the size and hardness of objects and estimate the grasping force. This novel approach circumvents the need for resource-intensive pre-processing steps, capitalizing on the simplicity and inherent parallelism of HDC operations. Our comprehensive analysis reveals that GraspHD surpasses state-of-the-art approaches in terms of overall classification accuracy. We have also implemented GraspHD on an FPGA to evaluate system efficiency. The results demonstrate that GraspHD operates at a speed 10x faster and offers an energy efficiency 26 times higher than existing learning algorithms, all while maintaining robust performance in noisy environments. These findings underscore the significant potential of GraspHD as a more efficient and effective solution for real-time robotic grasping applications.

show abstract

“…With the rapid development of artificial intelligence technology, scholars begin to apply deep learning in the field of part target recognition. Recognition algorithms based on machine vision technology and convolutional neural network emerge endlessly, including R-CNN, VGG, YOLO, Fast R-CNN, Faster R-CNN [1,2,3,4,5] , etc. Because of its high precision, high speed, no contact as well as no loss, it has gradually replaced the traditional feature-matching classification method.…”

Section: Introductionmentioning

confidence: 99%

Research on mechanical parts classification and recognition technology based on improved convolutional neural network YOLOv5

Xu,

Guo

2023

Third International Conference on Computer Graphics, Image, and Virtualization (ICCGIV 2023)

View full text Add to dashboard Cite

In the field of classification and recognition of mechanical parts, in order to solve the difficulties of small target detection, adhesion and overlapping parts classification and recognition, this paper proposes an automatic classification and recognition algorithm of mechanical parts based on improved convolutional neural network YOLOv5. The feature pyramid module PANet is added to the traditional YOLOv5 network structure to strengthen the characterization ability of the network. At the same time, NMS algorithm and image segmentation algorithm are used to improve the detection ability of network boundary segmentation, to improve the detection accuracy and recall rate. The experimental results show that both the traditional YOLOv5 and the improved YOLOv5 in this paper can realize the classification and identification of mechanical parts, and the improved YOLOv5 has better detection precision and accuracy, faster detection speed, and better network robustness.

show abstract

Pose estimation and robotic insertion tasks based on YOLO and layout features

Cited by 9 publications

References 25 publications

Extended residual learning with one-shot imitation learning for robotic assembly in semi-structured environment

Extended residual learning with one-shot imitation learning for robotic assembly in semi-structured environment

Efficient Event-Based Robotic Grasping Perception using Hyperdimensional Computing

Research on mechanical parts classification and recognition technology based on improved convolutional neural network YOLOv5

Contact Info

Product

Resources

About