Deep Learning Based Camera Pose Estimation in Multi-view Environment

Charco, Jorge L.; Vintimilla, Boris X.; Sappa, Ángel D.

doi:10.1109/sitis.2018.00041

Cited by 15 publications

(6 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Stereo vision [13,14] provides a convenient alternative to structured light methods, requiring only two regular cameras installed and calibrated on the same base platform. In a more general scheme, the cameras (possibly more than two) may be placed such that each captures a different view of the objects [15]. Stereo vision benefits from the availability of open-source software and extensive past research [16].…”

Section: Pose Estimationmentioning

confidence: 99%

Deep Bayesian-Assisted Keypoint Detection for Pose Estimation in Assembly Automation

Shi

Rahimpour

Ghafourian

et al. 2023

Sensors

View full text Add to dashboard Cite

Pose estimation is crucial for automating assembly tasks, yet achieving sufficient accuracy for assembly automation remains challenging and part-specific. This paper presents a novel, streamlined approach to pose estimation that facilitates automation of assembly tasks. Our proposed method employs deep learning on a limited number of annotated images to identify a set of keypoints on the parts of interest. To compensate for network shortcomings and enhance accuracy we incorporated a Bayesian updating stage that leverages our detailed knowledge of the assembly part design. This Bayesian updating step refines the network output, significantly improving pose estimation accuracy. For this purpose, we utilized a subset of network-generated keypoint positions with higher quality as measurements, while for the remaining keypoints, the network outputs only serve as priors. The geometry data aid in constructing likelihood functions, which in turn result in enhanced posterior distributions of keypoint pixel positions. We then employed the maximum a posteriori (MAP) estimates of keypoint locations to obtain a final pose, allowing for an update to the nominal assembly trajectory. We evaluated our method on a 14-point snap-fit dash trim assembly for a Ford Mustang dashboard, demonstrating promising results. Our approach does not require tailoring to new applications, nor does it rely on extensive machine learning expertise or large amounts of training data. This makes our method a scalable and adaptable solution for the production floors.

show abstract

Section: Pose Estimationmentioning

confidence: 99%

Deep Bayesian-Assisted Keypoint Detection for Pose Estimation in Assembly Automation

Shi

Rahimpour

Ghafourian

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…In order to overcome the limitation of obtaining real-world images with haze, 3D models of different scenarios are required in order to simulate realistic haze image datasets. It should be mentioned that the usage of 3D virtual environments to generate a dataset of synthetic images has already been considered for tackling different computer vision problems for instance object recognition (e.g., pedestrians (Fabbri et al, 2021), vehicles (Tang et al, 2019)), camera calibration (Charco et al, 2018;Charco et al, 2020;Charco et al, 2021), just to mention a few. In the current work, a similar strategy is followed to address the problem of image haze removal.…”

Section: Paired Real Images (Clear/haze)mentioning

confidence: 99%

“…The problem mentioned above, the lack of a large dataset for training deep learning based approaches, is a common problem in different image processing and computer vision approaches. This problem has been tackled by using images acquired in virtual 3D scenarios (e.g., (Fabbri et al, 2021;Tang et al, 2019;Charco et al, 2018). Images from these virtual scenarios, generally referred to as synthetic images, are used for training deep learning models.…”

Section: Related Workmentioning

confidence: 99%

Domain Adaptation in Image Dehazing: Exploring the Usage of Images From Virtual Scenarios

2022

Proceedings of the 16th International Conference on Computer Graphics, Visualization, Computer Vision and Image Processing (CGV

View full text Add to dashboard Cite

This work presents a novel domain adaptation strategy for deep learning-based approaches to solve the image haze removal problem. Firstly, a large set of synthetic images is generated by using a realistic 3D graphic simulator; these synthetic images contain different densities of haze, which are used for training the model that is later adapted to any real scenario. The adaptation process requires just a few images to fine-tune the model parameters. The proposed strategy allows overcoming the limitation of training a given model with few images. In other words, the proposed strategy implements the adaptation of a haze removal model trained with synthetic images to real scenarios. It should be noticed that it is quite difficult, if not impossible, to have large sets of pairs of real-world images (with and without haze) to train in a supervised way haze removal algorithms. Experimental results are provided showing the validity of the proposed domain adaptation strategy.

show abstract

“…In [10], researchers proposed a deep convolutional neural network with 2 m and 6°accuracy for large scale outdoor scenes and 0.5m and 10°for indoor scenes respectively. In both [11] and [12], the authors trained and tested the neural networks proposed in previous works on DTU dataset [13] which consists of images pairs of objects shot from different viewpoints, errors were reduced to a few centimeters.…”

Section: A Camera Pose Estimationmentioning

confidence: 99%

Siamese Convolutional Neural Network for Sub-millimeter-accurate Camera Pose Estimation and Visual Servoing

Cai

Pham

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Visual Servoing (VS), where images taken from a camera typically attached to the robot end-effector are used to guide the robot motions, is an important technique to tackle robotic tasks that require a high level of accuracy. We propose a new neural network, based on a Siamese architecture, for highly accurate camera pose estimation. This, in turn, can be used as a final refinement step following a coarse VS or, if applied in an iterative manner, as a standalone VS on its own. The key feature of our neural network is that it outputs the relative pose between any pair of images, and does so with submillimeter accuracy. We show that our network can reduce pose estimation errors to 0.6 mm in translation and 0.4 degrees in rotation, from initial errors of 10 mm / 5 degrees if applied once, or of several cm / tens of degrees if applied iteratively. The network can generalize to similar objects, is robust against changing lighting conditions, and to partial occlusions (when used iteratively). The high accuracy achieved enables tackling low-tolerance assembly tasks downstream: using our network, an industrial robot can achieve 97.5% success rate on a VGAconnector insertion task without any force sensing mechanism.

show abstract

Deep Learning Based Camera Pose Estimation in Multi-view Environment

Cited by 15 publications

References 12 publications

Deep Bayesian-Assisted Keypoint Detection for Pose Estimation in Assembly Automation

Deep Bayesian-Assisted Keypoint Detection for Pose Estimation in Assembly Automation

Domain Adaptation in Image Dehazing: Exploring the Usage of Images From Virtual Scenarios

Siamese Convolutional Neural Network for Sub-millimeter-accurate Camera Pose Estimation and Visual Servoing

Contact Info

Product

Resources

About