CosyPose: Consistent Multi-view Multi-object 6D Pose Estimation

Labbé, Yann; Carpentier, Justin; Aubry, Mathieu; Šivic, Josef

doi:10.1007/978-3-030-58520-4_34

Cited by 270 publications

(328 citation statements)

References 42 publications

Supporting

Mentioning

307

Contrasting

Order By: Relevance

“…Based on the results of the BOP challenge 2020 [ 26 ], the CosyPose method [ 20 ] had the best performance on the T-LESS (texture-less) dataset relevant to our problem. CosyPose allows for pose estimation from RGB images, however, it is unsuitable for use with our dataset due to its training data requirements.…”

Section: Resultsmentioning

confidence: 99%

“…Additionally, the training complexity is much higher in comparison with our method. In the original paper [ 20 ], the method was trained using 32 GPUs and the training took a considerable amount of time. The computational time for the pose estimation phase stated in the BOP leaderboards is approximately 500 ms, in this regard, we outperformed CosyPose almost three times.…”

Section: Resultsmentioning

confidence: 99%

“…Grasp learning in an end-to-end manner, without an intermediate representation of the object pose, is conceptually elegant, since end-to-end methods take an image as input, and the output is a corresponding pose. Such methods estimate the grasping position either directly [ 17 ] or in combination with other approaches [ 18 , 19 , 20 ]. However, generalization is an issue here, since these methods suffer from high demand for labeled data for training, and data acquisition is usually time-consuming and laborious.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Data-Driven Object Pose Estimation in a Practical Bin-Picking Application

Kozák

Sushkov

Kulich

et al. 2021

Sensors

View full text Add to dashboard Cite

This paper addresses the problem of pose estimation from 2D images for textureless industrial metallic parts for a semistructured bin-picking task. The appearance of metallic reflective parts is highly dependent on the camera viewing direction, as well as the distribution of light on the object, making conventional vision-based methods unsuitable for the task. We propose a solution using direct light at a fixed position to the camera, mounted directly on the robot’s gripper, that allows us to take advantage of the reflective properties of the manipulated object. We propose a data-driven approach based on convolutional neural networks (CNN), without the need for a hard-coded geometry of the manipulated object. The solution was modified for an industrial application and extensively tested in a real factory. Our solution uses a cheap 2D camera and allows for a semi-automatic data-gathering process on-site.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Data-Driven Object Pose Estimation in a Practical Bin-Picking Application

Kozák

Sushkov

Kulich

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…Pose estimation methods for household objects typically use a single RGB image [24,36], RGBD image [49], or separate setting for each [1,51]. Alternatively, recent work [29] uses multiple views to jointly predict multiple object poses, which achieves the best result on the YCB-Video dataset [51], T-LESS dataset [20], and BOP Challenge 2020 [21].…”

Section: Related Workmentioning

confidence: 99%

Learning Object Depth from Camera Motion and Video Object Segmentation

Griffin

Corso

2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

This paper addresses the problem of learning to estimate the depth of detected objects given some measurement of camera motion (e.g., from robot kinematics or vehicle odometry). We achieve this by 1) designing a recurrent neural network (DBox) that estimates the depth of objects using a generalized representation of bounding boxes and uncalibrated camera movement and 2) introducing the Object Depth via Motion and Detection Dataset (ODMD). ODMD training data are extensible and configurable, and the ODMD benchmark includes 21,600 examples across four validation and test sets. These sets include mobile robot experiments using an end-effector camera to locate objects from the YCB dataset and examples with perturbations added to camera motion or bounding box data. In addition to the ODMD benchmark, we evaluate DBox in other monocular application domains, achieving state-of-the-art results on existing driving and robotics benchmarks and estimating the depth of objects using a camera phone.

show abstract

“…Using iterative closest point(ICP) as a refinement phase PoseCNN makes their 6D pose estimation accurate. In [6], a method named CosyPose was proposed, which uses multiple views to reconstruct a scene composed of multiple objects to estimate 6D pose in three stages. In the first stage, for each view, initial object candidates are estimated separately.…”

Section: Introductionmentioning

confidence: 99%

6D Pose Estimation for Subsea Intervention in Turbid Waters

et al. 2021

View full text Add to dashboard Cite

Manipulation tasks on subsea instalments require extremely precise detection and localization of objects of interest. This problem is referred to as “pose estimation”. In this work, we present a framework for detecting and predicting 6DoF pose for relevant objects (fish-tail, gauges, and valves) on a subsea panel under varying water turbidity. A deep learning model that takes 3D vision data as an input is developed, providing a more robust 6D pose estimate. Compared to the 2D vision deep learning model, the proposed method reduces rotation and translation prediction error by (−Δ0.39∘) and translation (−Δ6.5 mm), respectively, in high turbid waters. The proposed approach is able to provide object detection as well as 6D pose estimation with an average precision of 91%. The 6D pose estimation results show 2.59∘ and 6.49 cm total average deviation in rotation and translation as compared to the ground truth data on varying unseen turbidity levels. Furthermore, our approach runs at over 16 frames per second and does not require pose refinement steps. Finally, to facilitate the training of such model we also collected and automatically annotated a new underwater 6D pose estimation dataset spanning seven levels of turbidity.

show abstract

CosyPose: Consistent Multi-view Multi-object 6D Pose Estimation

Cited by 270 publications

References 42 publications

Data-Driven Object Pose Estimation in a Practical Bin-Picking Application

Data-Driven Object Pose Estimation in a Practical Bin-Picking Application

Learning Object Depth from Camera Motion and Video Object Segmentation

6D Pose Estimation for Subsea Intervention in Turbid Waters

Contact Info

Product

Resources

About