ROFT: Real-Time Optical Flow-Aided 6D Object Pose and Velocity Tracking

Piga, Nicola A.; Onyshchuk, Yuriy; Pasquale, Giulia; Pattacini, Ugo; Natale, Lorenzo

doi:10.1109/lra.2021.3119379

Cited by 16 publications

(5 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to those render-and-compare algorithms, PoseRBPF [14] uses a Rao-Blackwellized particle filter and pose-representative latent codes [44]. Also, TP-AE [45] proposed a temporally primed framework with auto encoders, while ROFT [46] synchronizes low framerate pose estimates with fast optical flow.…”

Section: Related Workmentioning

confidence: 99%

A Sparse Gaussian Approach to Region-Based 6DoF Object Tracking

Stoiber

Pfanne

Strobl

et al. 2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

In many applications of advanced robotic manipulation, six degrees of freedom (6DoF) object pose estimates are continuously required. In this work, we develop a multimodality tracker that fuses information from visual appearance and geometry to estimate object poses. The algorithm extends our previous method ICG, which uses geometry, to additionally consider surface appearance. In general, object surfaces contain local characteristics from text, graphics, and patterns, as well as global differences from distinct materials and colors. To incorporate this visual information, two modalities are developed. For local characteristics, keypoint features are used to minimize distances between points from keyframes and the current image. For global differences, a novel region approach is developed that considers multiple regions on the object surface. In addition, it allows the modeling of external geometries. Experiments on the YCB-Video and OPT datasets demonstrate that our approach ICG+ performs best on both datasets, outperforming both conventional and deep learningbased methods. At the same time, the algorithm is highly efficient and runs at more than 300 Hz. The source code of our tracker is publicly available.

show abstract

Section: Related Workmentioning

confidence: 99%

A Sparse Gaussian Approach to Region-Based 6DoF Object Tracking

Stoiber

Pfanne

Strobl

et al. 2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…[26] integrates offthe-shelf segmentation convolutional neural networks, multihypothesis point cloud registration and a Kalman Filter to track the object pose and velocity. A follow up work [29] leverages real-time optical flow to improve tracking performance in fast motion. In [27], for each RGB-D frame, the segmentation mask of the object of interest is combined with the depth frame to produce a partial point cloud of the object which is further refined by outlier removal.…”

Section: Related Workmentioning

confidence: 99%

An Improved Approach to 6D Object Pose Tracking in Fast Motion Scenarios

Vandewalle

Slaets

et al. 2022

2022 Sixth IEEE International Conference on Robotic Computing (IRC)

View full text Add to dashboard Cite

Tracking 6D poses of objects in video sequences is important for many applications such as robot manipulation and augmented reality. End-to-end deep learning based 6D pose tracking methods have achieved notable performance both in terms of accuracy and speed on standard benchmarks characterized by slowly varying poses. However, these methods fail to address a key challenge for using 6D pose trackers in fast motion scenarios. The performance of temporal trackers degrades significantly in fast motion scenarios and tracking failures occur frequently. In this work, we propose a framework to make endto-end 6D pose trackers work better for fast motion scenarios. We integrate the "Relative Pose Estimation Network" from an end-to-end 6D pose tracker into an EKF framework. The EKF adopts a constant velocity motion model and its measurement is computed from the output of the "Relative Pose Estimation Network". The proposed method is evaluated on challenging hand-object interaction sequences from the Laval dataset and compared against the original end-to-end pose tracker, referred to as the baseline. Experiments show that integration with EKF significantly improves the tracking performance, achieving a pose detection rate of 85.23% compared to 61.32% achieved by the baseline. The proposed framework exceeds the real-time performance requirement of 30 fps.

show abstract

“…[39] and recently Ref. [40] introduced variational and CNN-based methods for flow-aided pose estimation, based on fulfilled brightness assumption. Nevertheless, an automatic and light resistant flow-based pose-estimation method that works correspondence-free, and takes geometrical, textural and coherent scene motion into account has never been addressed before.…”

Section: Related Workmentioning

confidence: 99%

INV-Flow2PoseNet: Light-Resistant Rigid Object Pose from Optical Flow of RGB-D Images Using Images, Normals and Vertices

Fetzer

Reis²,

Stricker³

2022

Sensors

View full text Add to dashboard Cite

This paper presents a novel architecture for simultaneous estimation of highly accurate optical flows and rigid scene transformations for difficult scenarios where the brightness assumption is violated by strong shading changes. In the case of rotating objects or moving light sources, such as those encountered for driving cars in the dark, the scene appearance often changes significantly from one view to the next. Unfortunately, standard methods for calculating optical flows or poses are based on the expectation that the appearance of features in the scene remains constant between views. These methods may fail frequently in the investigated cases. The presented method fuses texture and geometry information by combining image, vertex and normal data to compute an illumination-invariant optical flow. By using a coarse-to-fine strategy, globally anchored optical flows are learned, reducing the impact of erroneous shading-based pseudo-correspondences. Based on the learned optical flows, a second architecture is proposed that predicts robust rigid transformations from the warped vertex and normal maps. Particular attention is paid to situations with strong rotations, which often cause such shading changes. Therefore, a 3-step procedure is proposed that profitably exploits correlations between the normals and vertices. The method has been evaluated on a newly created dataset containing both synthetic and real data with strong rotations and shading effects. These data represent the typical use case in 3D reconstruction, where the object often rotates in large steps between the partial reconstructions. Additionally, we apply the method to the well-known Kitti Odometry dataset. Even if, due to fulfillment of the brightness assumption, this is not the typical use case of the method, the applicability to standard situations and the relation to other methods is therefore established.

show abstract

ROFT: Real-Time Optical Flow-Aided 6D Object Pose and Velocity Tracking

Cited by 16 publications

References 30 publications

A Sparse Gaussian Approach to Region-Based 6DoF Object Tracking

A Sparse Gaussian Approach to Region-Based 6DoF Object Tracking

An Improved Approach to 6D Object Pose Tracking in Fast Motion Scenarios

INV-Flow2PoseNet: Light-Resistant Rigid Object Pose from Optical Flow of RGB-D Images Using Images, Normals and Vertices

Contact Info

Product

Resources

About