Monocular Object Orientation Estimation using Riemannian Regression and Classification Networks

Mahendran, Siddharth; Lu, Ming; Haider, Ali; Vidal, René

doi:10.48550/arxiv.1807.07226

Cited by 3 publications

(3 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…4 (c) shows an example. CNNs have been employed for 3D pose estimation from a single image [56]. Simultaneous depth and 3D orientation estimation of optical microrobots can be achieved via CNNs as well [53] (see Fig.…”

Section: Machine Learning-based Methodsmentioning

confidence: 99%

Fabrication and optical manipulation of micro-robots for biomedical applications

Zhang

Ren

Barbot

et al. 2022

Matter

View full text Add to dashboard Cite

Section: Machine Learning-based Methodsmentioning

confidence: 99%

Fabrication and optical manipulation of micro-robots for biomedical applications

Zhang

Ren

Barbot

et al. 2022

Matter

View full text Add to dashboard Cite

“…Several clear avenues for improvement exist. Predictions might be made on a pixel-wise basis [13] to improve spatial accuracy, and pose binning [44] might improve accuracy. The Cross Entropy Method could sample around proposals for assessment with a Q function [12].…”

Section: B Physical Implications and Future Workmentioning

confidence: 99%

The CoSTAR Block Stacking Dataset: Learning with Workspace Constraints

Hundt

Jain

Lin

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

A robot can now grasp an object more effectively than ever before, but once it has the object what happens next? We show that a mild relaxation of the task and workspace constraints implicit in existing object grasping datasets can cause neural network based grasping algorithms to fail on even a simple block stacking task when executed under more realistic circumstances.To address this, we introduce the JHU CoSTAR Block Stacking Dataset (BSD), where a robot interacts with 5.1 cm colored blocks to complete an order-fulfillment style block stacking task. It contains dynamic scenes and real time-series data in a less constrained environment than comparable datasets. There are nearly 12,000 stacking attempts and over 2 million frames of real data. We discuss the ways in which this dataset provides a valuable resource for a broad range of other topics of investigation.We find that hand-designed neural networks that work on prior datasets do not generalize to this task. Thus, to establish a baseline for this dataset, we demonstrate an automated search of neural network based models using a novel multiple-input HyperTree MetaModel, and find a final model which makes reasonable 3D pose predictions for grasping and stacking on our dataset.The CoSTAR BSD, code, and instructions are available at sites.google.com/site/costardataset.

show abstract

“…For example, CNNs have been employed for the estimation of the 3D pose of an object from a single image. 25 A CNN-based method for estimating the 3D pose and depth of optically transparent microrobots has been developed. 26 However, for a neural network to be able to estimate the 3D pose of different microrobots, a large database should be collected, which consists of microscopic images of different microrobots with the combination of different depth values and poses.…”

mentioning

confidence: 99%

Data-Driven Microscopic Pose and Depth Estimation for Optical Microrobot Manipulation

Zhang

Zheng

et al. 2020

ACS Photonics

View full text Add to dashboard Cite

Optical microrobots have a wide range of applications in biomedical research for both in vitro and in vivo studies. In most microrobotic systems, the video captured by a monocular camera is the only way for capturing the movements of microrobots, and only planar motion can be captured by a monocular camera system. Accurate depth estimation is essential for the 3D reconstruction or autofocusing of micro-platforms, while the pose and depth estimation are necessary to enhance the 3D perception of the micro-robotic systems to enable dexterous micro-manipulation and other tasks. In this paper, we propose a data-driven method for pose and depth estimation in an optically manipulated microrobotic system. Focus measurement is used to obtain features for Gaussian Process Regression (GPR), which enables precise depth estimation. For mobile microrobots with varying poses, a novel method is developed based on a deep residual neural network with the incorporation of prior domain knowledge about the optical microrobots encoded via GPR. The method can simultaneously 1 track microrobots with complex shapes and estimate the pose and depth values of the optical microrobots.Cross-validation has been conducted to demonstrate the sub-micron accuracy of the proposed method and precise pose and depth perception for microrobots. We further demonstrate the generalisability of the method by adapting it to microrobots of different shapes using transfer learning with few-shot calibration. Intuitive visualization is provided to facilitate effective human-robot interaction during micro-manipulation based on pose and depth estimation results.

show abstract

Monocular Object Orientation Estimation using Riemannian Regression and Classification Networks

Cited by 3 publications

References 25 publications

Fabrication and optical manipulation of micro-robots for biomedical applications

Fabrication and optical manipulation of micro-robots for biomedical applications

The CoSTAR Block Stacking Dataset: Learning with Workspace Constraints

Data-Driven Microscopic Pose and Depth Estimation for Optical Microrobot Manipulation

Contact Info

Product

Resources

About