6D Pose Estimation of Objects: Recent Technologies and Challenges

He, Zaixing; Feng, Wang; Zhao, Xinyue; Lv, Yongfeng

doi:10.3390/app11010228

Cited by 39 publications

(23 citation statements)

References 88 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The pose of an object is a critical indicator of the state of the object. For dynamic objects, their poses are constantly changing and it is more difficult to estimate than static objects (Ding et al, 2018;He et al, 2020;Tao et al, 2021). For vehicles, the process of changing pose represents the interaction between the vehicle and the environment, which is very important for the vehicle to perceive the environment.…”

Section: Introductionmentioning

confidence: 99%

A siamese network-based approach for vehicle pose estimation

Zhao

Tao

Huang

et al. 2022

Front. Bioeng. Biotechnol.

View full text Add to dashboard Cite

We propose a deep learning-based vehicle pose estimation method based on a monocular camera called FPN PoseEstimateNet. The FPN PoseEstimateNet consists of a feature extractor and a pose calculate network. The feature extractor is based on Siamese network and a feature pyramid network (FPN) is adopted to deal with feature scales. Through the feature extractor, a correlation matrix between the input images is obtained for feature matching. With the time interval as the label, the feature extractor can be trained independently of the pose calculate network. On the basis of the correlation matrix and the standard matrix, the vehicle pose changes can be predicted by the pose calculate network. Results show that the network runs at a speed of 6 FPS, and the parameter size is 101.6 M. In different sequences, the angle error is within 8.26° and the maximum translation error is within 31.55 m.

show abstract

Section: Introductionmentioning

confidence: 99%

A siamese network-based approach for vehicle pose estimation

Zhao

Tao

Huang

et al. 2022

Front. Bioeng. Biotechnol.

View full text Add to dashboard Cite

show abstract

“…To accomplish this, there is the need to develop a robust algorithm capable of estimating the object's pose inside the 360°frame. 6DoF estimation is one of the main challenging research topics in computer vision [12][13][14].…”

Section: Introductionmentioning

confidence: 99%

Object Pose Detection to Enable 3D Interaction from 2D Equirectangular Images in Mixed Reality Educational Settings

et al. 2022

View full text Add to dashboard Cite

In this paper, we address the challenge of estimating the 6DoF pose of objects in 2D equirectangular images. This solution allows the transition to the objects’ 3D model from their current pose. In particular, it finds application in the educational use of 360° videos, where it enhances the learning experience of students by making it more engaging and immersive due to the possible interaction with 3D virtual models. We developed a general approach usable for any object and shape. The only requirement is to have an accurate CAD model, even without textures of the item, whose pose must be estimated. The developed pipeline has two main steps: vehicle segmentation from the image background and estimation of the vehicle pose. To accomplish the first task, we used deep learning methods, while for the second, we developed a 360° camera simulator in Unity to generate synthetic equirectangular images used for comparison. We conducted our tests using a miniature truck model whose CAD was at our disposal. The developed algorithm was tested using a metrological analysis applied to real data. The results showed a mean difference of 1.5 with a standard deviation of 1 from the ground truth data for rotations, and 1.4 cm with a standard deviation of 1.5 cm for translations over a research range of ±20 and ±20 cm, respectively.

show abstract

“…However, these methods struggle in cluttered environments subjected to partial occlusion or object truncation. More information on such methods can be found in reviews [15,16]. In recent times, CNN-based methods have shown robustness to the challenges faced by the traditional methods, and they have been developed to enable pose estimation from a single image.…”

Section: Introductionmentioning

confidence: 99%

Geometric Change Detection in Digital Twins

Tiril

Maria

Rasheed

et al. 2021

Digital

View full text Add to dashboard Cite

Digital twins are meant to bridge the gap between real-world physical systems and virtual representations. Both stand-alone and descriptive digital twins incorporate 3D geometric models, which are the physical representations of objects in the digital replica. Digital twin applications are required to rapidly update internal parameters with the evolution of their physical counterpart. Due to an essential need for having high-quality geometric models for accurate physical representations, the storage and bandwidth requirements for storing 3D model information can quickly exceed the available storage and bandwidth capacity. In this work, we demonstrate a novel approach to geometric change detection in a digital twin context. We address the issue through a combined solution of dynamic mode decomposition (DMD) for motion detection, YOLOv5 for object detection, and 3D machine learning for pose estimation. DMD is applied for background subtraction, enabling detection of moving foreground objects in real-time. The video frames containing detected motion are extracted and used as input to the change detection network. The object detection algorithm YOLOv5 is applied to extract the bounding boxes of detected objects in the video frames. Furthermore, we estimate the rotational pose of each object in a 3D pose estimation network. A series of convolutional neural networks (CNNs) conducts feature extraction from images and 3D model shapes. Then, the network outputs the camera orientation’s estimated Euler angles concerning the object in the input image. By only storing data associated with a detected change in pose, we minimize necessary storage and bandwidth requirements while still recreating the 3D scene on demand. Our assessment of the new geometric detection framework shows that the proposed methodology could represent a viable tool in emerging digital twin applications.

show abstract

6D Pose Estimation of Objects: Recent Technologies and Challenges

Cited by 39 publications

References 88 publications

A siamese network-based approach for vehicle pose estimation

A siamese network-based approach for vehicle pose estimation

Object Pose Detection to Enable 3D Interaction from 2D Equirectangular Images in Mixed Reality Educational Settings

Geometric Change Detection in Digital Twins

Contact Info

Product

Resources

About