Robotic Perception of Transparent Objects: A Review

Jiang, Jiaqi; Cao, Guanqun; Deng, Jiankang; Do, Thanh-Toan; Luo, Shan

doi:10.1109/tai.2023.3326120

Cited by 6 publications

(4 citation statements)

References 125 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the 3D coordinates X and Y are usually not integers, in the 2D image, the V value must be distributed among the surrounding four 2D pixels (Figure 1). Real gaps that exist due to physical limitations of the camera technology [1,2] used may not be fully considered (A). If neighboring 3D X/Y coordinates are further than one pixel apart, then false gaps between these pixels in the 2D image are created (B).…”

Section: Introduction 1challenges In Single-shot Projection Methodsmentioning

confidence: 99%

“…For example, valid gaps, such as car windows, are not considered (see challenge (A), Figure 1). (Transparent objects are a challenge for conventional sensors in the visible spectrum [2]. Transparency-conscious consideration is imperative and a corner case in the stereo image [1,36].)…”

Section: Qualitative and Quantitative Experiments 61 Density And Accu...mentioning

confidence: 99%

“…[64]). This aspect of the different modalities is becoming increasingly important, for example, in the detection of optically uncooperative objects in the visual spectral range [2,42] or in trusted human-robot interactions [13,14].…”

Section: Computation Time Memory Usage and Complexity Classmentioning

confidence: 99%

“…Table A1 shows an overview of the algorithms. These algorithms describe subprocesses for our TMRP (top) and the state-of-the-art (SOTA) projection (bottom) procedure; see Algorithm 1. if point_list.length > 2 then triangle_list append (point_list[0], point_list [1], point_list [2]) end if if point_list.length = 4 then triangle_list append (point_list [1], point_list [2], point_list [3]) triangle_list append (point_list [2], point_list [3], point_list[0]) triangle_list append (point_list [3], point_list[0], point_list…”

Section: Supplementary Materialsmentioning

confidence: 99%

See 3 more Smart Citations

Triangle-Mesh-Rasterization-Projection (TMRP): An Algorithm to Project a Point Cloud onto a Consistent, Dense and Accurate 2D Raster Image

Junger

Buch

Notni

2023

Sensors

View full text Add to dashboard Cite

The projection of a point cloud onto a 2D camera image is relevant in the case of various image analysis and enhancement tasks, e.g., (i) in multimodal image processing for data fusion, (ii) in robotic applications and in scene analysis, and (iii) for deep neural networks to generate real datasets with ground truth. The challenges of the current single-shot projection methods, such as simple state-of-the-art projection, conventional, polygon, and deep learning-based upsampling methods or closed source SDK functions of low-cost depth cameras, have been identified. We developed a new way to project point clouds onto a dense, accurate 2D raster image, called Triangle-Mesh-Rasterization-Projection (TMRP). The only gaps that the 2D image still contains with our method are valid gaps that result from the physical limits of the capturing cameras. Dense accuracy is achieved by simultaneously using the 2D neighborhood information (rx,ry) of the 3D coordinates in addition to the points P(X,Y,V). In this way, a fast triangulation interpolation can be performed. The interpolation weights are determined using sub-triangles. Compared to single-shot methods, our algorithm is able to solve the following challenges. This means that: (1) no false gaps or false neighborhoods are generated, (2) the density is XYZ independent, and (3) ambiguities are eliminated. Our TMRP method is also open source, freely available on GitHub, and can be applied to almost any sensor or modality. We also demonstrate the usefulness of our method with four use cases by using the KITTI-2012 dataset or sensors with different modalities. Our goal is to improve recognition tasks and processing optimization in the perception of transparent objects for robotic manufacturing processes.

show abstract

Section: Introduction 1challenges In Single-shot Projection Methodsmentioning

confidence: 99%

Section: Qualitative and Quantitative Experiments 61 Density And Accu...mentioning

confidence: 99%

Section: Computation Time Memory Usage and Complexity Classmentioning

confidence: 99%

Section: Supplementary Materialsmentioning

confidence: 99%

See 2 more Smart Citations

Triangle-Mesh-Rasterization-Projection (TMRP): An Algorithm to Project a Point Cloud onto a Consistent, Dense and Accurate 2D Raster Image

Junger

Buch

Notni

2023

Sensors

View full text Add to dashboard Cite

show abstract

Dynamic Informed Bias RRT*-Connect: Improving Heuristic Guidance by Dynamic Informed Bias Using Hybrid Dual Trees Search

Li,

Kang,

Han

2024

J Intell Robot Syst

View full text Add to dashboard Cite

The RRT*-Connect algorithm enhances efficiency through dual tree bias growth, yet this bias can be inherently blind, potentially affecting the algorithm’s heuristic performance. In contrast, the Informed RRT* algorithm narrows the planning problem’s scope by leveraging an informed region, thereby improving convergence efficiency towards optimal solutions. However, this approach relies on the prior establishment of feasible paths. Combining these two algorithms can address the challenges posed by Informed RRT while also accelerating convergence towards optimality, albeit without resolving the issue of blind bias in dual trees.In this paper, we proposed a novel algorithm: Dynamic Informed Bias RRT*-Connect. This algorithm, grounded in potential and explicit informed bias sampling, introduces a dynamical bias points set that guides dual tree growth with precision objectives. Additionally, we enhance the evaluation framework for algorithmic heuristics by introducing two innovative metrics that effectively capture the algorithm’s characteristics. The improvements observed in traditional indicators demonstrate that the proposed algorithm exhibits greater heuristic compared to RRT*-Connect and Informed RRT*-Connect. These findings also suggest the viability of the new metrics introduced in our evaluation framework.

show abstract

RFTrans: Leveraging Refractive Flow of Transparent Objects for Surface Normal Estimation and Manipulation

Tang,

Liu,

Zhang

et al. 2024

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Robotic Perception of Transparent Objects: A Review

Cited by 6 publications

References 125 publications

Triangle-Mesh-Rasterization-Projection (TMRP): An Algorithm to Project a Point Cloud onto a Consistent, Dense and Accurate 2D Raster Image

Triangle-Mesh-Rasterization-Projection (TMRP): An Algorithm to Project a Point Cloud onto a Consistent, Dense and Accurate 2D Raster Image

Dynamic Informed Bias RRT*-Connect: Improving Heuristic Guidance by Dynamic Informed Bias Using Hybrid Dual Trees Search

RFTrans: Leveraging Refractive Flow of Transparent Objects for Surface Normal Estimation and Manipulation

Contact Info

Product

Resources

About