Industrial part localization and grasping using a robotic arm guided by 2D monocular vision

Zheng, Zhaohui; Ma, Yong; Zheng, Hong; Gu, Yu; Li, Mingyu

doi:10.1108/ir-06-2018-0128

Cited by 21 publications

(11 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Saxena et al (2010)'s work on grasping objects in cluttered environments acknowledges the problem of accurately sensing a complex 3D environment, but attempts to avoid it by storing prebuilt 3D models and using them to better analyze a single stereovision image rather than by collecting more information. In a similar vein, Zheng et al (2018) approaches industrial grasping by trying to more accurately map known features to objects instead of by trying to collect more data to resolve ambiguities in the images. A typical approach in literature is to train a neural network to produce grasps by annotating individual images with grasp candidates.…”

Section: Related Workmentioning

confidence: 99%

Aiding Grasp Synthesis for Novel Objects Using Heuristic-Based and Data-Driven Active Vision Methods

2021

View full text Add to dashboard Cite

In this work, we present several heuristic-based and data-driven active vision strategies for viewpoint optimization of an arm-mounted depth camera to aid robotic grasping. These strategies aim to efficiently collect data to boost the performance of an underlying grasp synthesis algorithm. We created an open-source benchmarking platform in simulation (https://github.com/galenbr/2021ActiveVision), and provide an extensive study for assessing the performance of the proposed methods as well as comparing them against various baseline strategies. We also provide an experimental study with a real-world two finger parallel jaw gripper setup by utilizing an existing grasp planning benchmark in the literature. With these analyses, we were able to quantitatively demonstrate the versatility of heuristic methods that prioritize certain types of exploration, and qualitatively show their robustness to both novel objects and the transition from simulation to the real world. We identified scenarios in which our methods did not perform well and objectively difficult scenarios, and present a discussion on which avenues for future research show promise.

show abstract

Section: Related Workmentioning

confidence: 99%

Aiding Grasp Synthesis for Novel Objects Using Heuristic-Based and Data-Driven Active Vision Methods

2021

View full text Add to dashboard Cite

show abstract

“…Machine vision has been widely used in industrial robotic manufacturing systems because of its fast, stable and noncontact characteristics (Zheng et al, 2018). Mounting the camera near the end-effector of the robot arm can form a handeye vision system, which assists the robot in the threedimensional positioning of the target object through visual guidance (Maruyama et al, 1990).…”

Section: Related Workmentioning

confidence: 99%

Hybrid calibration and detection approach for mobile robotic manufacturing systems

Bai

Zhang

Qin

et al. 2020

View full text Add to dashboard Cite

Purpose The purpose of this paper is to present a visual detection approach to predict the poses of target objects placed in arbitrary positions before completing the corresponding tasks in mobile robotic manufacturing systems. Design/methodology/approach A hybrid visual detection approach that combines monocular vision and laser ranging is proposed based on an eye-in-hand vision system. The laser displacement sensor is adopted to achieve normal alignment for an arbitrary plane and obtain depth information. The monocular camera measures the two-dimensional image information. In addition, a robot hand-eye relationship calibration method is presented in this paper. Findings First, a hybrid visual detection approach for mobile robotic manufacturing systems is proposed. This detection approach is based on an eye-in-hand vision system consisting of one monocular camera and three laser displacement sensors and it can achieve normal alignment for an arbitrary plane and spatial positioning of the workpiece. Second, based on this vision system, a robot hand-eye relationship calibration method is presented and it was successfully applied to a mobile robotic manufacturing system designed by the authors’ team. As a result, the relationship between the workpiece coordinate system and the end-effector coordinate system could be established accurately. Practical implications This approach can quickly and accurately establish the relationship between the coordinate system of the workpiece and that of the end-effector. The normal alignment accuracy of the hand-eye vision system was less than 0.5° and the spatial positioning accuracy could reach 0.5 mm. Originality/value This approach can achieve normal alignment for arbitrary planes and spatial positioning of the workpiece and it can quickly establish the pose relationship between the workpiece and end-effector coordinate systems. Moreover, the proposed approach can significantly improve the work efficiency, flexibility and intelligence of mobile robotic manufacturing systems.

show abstract

“…Doerr et al used ORB features to recognize and classify various industrial parts [14]. Zheng et al used ORB features to locate and grab mechanical parts [15]. As long as been localized, the parts can be adjusted and assembled by driving the motion modules, such as motion platforms or robots, in the assembly systems.…”

Section: Introductionmentioning

confidence: 99%

A Coaxial Alignment Method for Large Flange Parts Assembly Using Multiple Local Images

Sun

Zhang

2021

IEEE Access

View full text Add to dashboard Cite

The large flange parts in aero-engines are usually manually assembled. Collision damage caused by assembly alignment error often occurs in the assembly process, which affects the assembly accuracy and product reliability. The machine vision-based alignment methods usually achieve the highprecision measurement of the parts by obtaining the high-resolution images of the whole parts with a combination of laser distance sensors. Hence, existing methods are high costly and inefficient. In this paper, a new alignment method based on the principle of coaxial alignment for large flange parts is proposed. The proposed method first obtains multiple high-precision image pairs from the local field of views at the fitting surfaces of flanges. The clearance and bolt holes in each image pair are then extracted via edge recognition and Hough Transformation. Two optimization models are built to calculate the translation adjustment and rotation adjustment. The optimization model 1 is built with the translation adjustments of the flange as the variables and the consistency of the clearances as the objective function. The algorithm to solve the model based on the gradient-descent method is proposed. The positions of the bolt holes in the images are subsequently adjusted based on the translation adjustment, and the rotation adjustment is calculated by solving the built optimization model 2. The experiments show that the proposed method can be applied to the assembly process of large flange parts, and the visual servo control model based on this method also has good stability.

show abstract

Industrial part localization and grasping using a robotic arm guided by 2D monocular vision

Cited by 21 publications

References 20 publications

Aiding Grasp Synthesis for Novel Objects Using Heuristic-Based and Data-Driven Active Vision Methods

Aiding Grasp Synthesis for Novel Objects Using Heuristic-Based and Data-Driven Active Vision Methods

Hybrid calibration and detection approach for mobile robotic manufacturing systems

A Coaxial Alignment Method for Large Flange Parts Assembly Using Multiple Local Images

Contact Info

Product

Resources

About