3D Object Pose Estimation for Robotic Packing Applications

Rodríguez-Garavito, C. H.; Camacho-Munoz, Guillermo A.; Álvarez‐Martínez, David; Cardenas, Karol Viviana; Rojas, David Mateo; Grimaldos, Andrés

doi:10.1007/978-3-030-00353-1_40

Cited by 7 publications

(7 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The levels of the algorithm factor are denoted as A wpk and A woutpk (wpk and woutpk for "with previous knowledge" and "without previous knowledge", respectively). Both algorithms are based on previously reported incremental modeling algorithms [5,7], to which stages have been added for (1) plane detection from point clouds [28], (2) pose estimation from convex hull model analysis [8], and (3) integration of available prior knowledge of box sizes in the scene. Additionally, a new strategy has been proposed for creating the global detection map, including routines to (a) remove elements from the global map that have exited the consolidation area, (b) eliminate redundant plane segments from multiple mappings…”

Section: Rgbd Slam 6d Plane Segment Tracking Algorithm Endmentioning

confidence: 99%

See 1 more Smart Citation

Evaluation of the use of box size priors for 6D plane segment tracking from point clouds with applications in cargo packing

Muñoz,

Nope-Rodríguez,

Loaiza-Correa

et al. 2024

Preprint

View full text Add to dashboard Cite

Available solutions to assist human operators in cargo packing processes offer alternatives to maximize the spatial occupancy of containers used in intralogistics. However, these solutions consist of sequential instructions for picking each box and positioning it in the containers, making it challenging for an operator to interpret and requiring them to alternate between reading the instructions and executing the task. A potential solution to these issues lies in a tool that naturally communicates each box's initial and final location in the desired sequence to the operator. While 6D visual object tracking systems have demonstrated good performance, they have yet to be evaluated in real-world scenarios of manual box packing. They also need to use the available prior knowledge of the packing operation, such as the number of boxes, box size, and physical packing sequence. This study explores the inclusion of box size priors in 6D plane segment tracking systems driven by images from moving cameras and quantifies their contribution in terms of tracker performance when assessed in manual box packing operations. To do this, it compares the performance of a plane segment tracking system, considering variations in the tracking algorithm and camera speed (onboard the packing operator) during the mapping of a manual cargo packing process. The tracking algorithm varies at two levels: algorithm (Awpk), which integrates prior knowledge of box sizes in the scene, and algorithm (Awoutpk), which assumes ignorance of box properties. Camera speed is also evaluated at two levels: low speed (Slow) and high speed (Shigh). This study analyzes the impact of these factors on the precision, recall, and F1-score of the plane segment tracking system. ANOVA analysis was applied to the precision and F1-score results, which allows determining that neither the camera speed-algorithm interactions nor the camera speed are significant in the precision of the tracking system. The factor that presented a significant effect is the tracking algorithm. Tukey's pairwise comparisons concluded that the precision and F1-score of each algorithm level are significantly different, with algorithm Awpk being superior in each evaluation. This superiority reaches its maximum in the tracking of top plane segments: 22 and 14 percentage units for precision and F1-score metrics, respectively. However, the results on the recall metric remain similar with and without the addition of prior knowledge. The contribution of including prior knowledge of box sizes in (6D) plane segment tracking algorithms is identified in reducing false positives. This reduction is associated with significant increases in the tracking system's precision and F1-score metrics. Future work will investigate whether the identified benefits propagate to the tracking problem on objects composed of plane segments, such as cubes or boxes.

show abstract

Section: Rgbd Slam 6d Plane Segment Tracking Algorithm Endmentioning

confidence: 99%

“…2. To obtain an updated object map in a dynamic environment with continuous repositioning and removing boxes from the working area, especially when using pose estimation techniques based on a single image [8][9][10] or techniques that do not remove objects that have exited the scene [11,12]. 3.…”

Section: Introductionmentioning

confidence: 99%

Evaluation of the use of box size priors for 6D plane segment tracking from point clouds with applications in cargo packing

Muñoz,

Nope-Rodríguez,

Loaiza-Correa

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…A point cloud acquired from RGB-D images or LIDAR has also been used for cuboid detection [7,8,9,10,11,12,13,14,25,26,27]. Shape descriptors for arbitrary 3D objects were proposed for object classification, including cuboids [7,8,12].…”

Section: Related Workmentioning

confidence: 99%

“…Shape descriptors for arbitrary 3D objects were proposed for object classification, including cuboids [7,8,12]. For indoor environments, prior knowledge of a room layout was incorporated to globally optimize object arrangement including cuboids in the room [9,12,27]. To detect buildings as cuboids, a closed polyhedral model is searched from planes detected in a point cloud [25].…”

Section: Related Workmentioning

confidence: 99%

Incremental 3D Cuboid Modeling with Drift Compensation

Mishima

Uchiyama

Thomas

et al. 2019

Sensors

View full text Add to dashboard Cite

This paper presents a framework of incremental 3D cuboid modeling by using the mapping results of an RGB-D camera based simultaneous localization and mapping (SLAM) system. This framework is useful in accurately creating cuboid CAD models from a point cloud in an online manner. While performing the RGB-D SLAM, planes are incrementally reconstructed from a point cloud in each frame to create a plane map. Then, cuboids are detected in the plane map by analyzing the positional relationships between the planes, such as orthogonality, convexity, and proximity. Finally, the position, pose, and size of a cuboid are determined by computing the intersection of three perpendicular planes. To suppress the false detection of the cuboids, the cuboid shapes are incrementally updated with sequential measurements to check the uncertainty of the cuboids. In addition, the drift error of the SLAM is compensated by the registration of the cuboids. As an application of our framework, an augmented reality-based interactive cuboid modeling system was developed. In the evaluation at cluttered environments, the precision and recall of the cuboid detection were investigated, compared with a batch-based cuboid detection method, so that the advantages of our proposed method were clarified.

show abstract

“…For example, it has been used in 3D reconstruction for object modeling [ 10 , 11 , 12 , 13 ] and indoor scenes [ 14 , 15 ] and mobile robots’ navigation and mapping [ 16 , 17 , 18 , 19 ]. The industrial applications include palletizing tasks [ 20 , 21 ], safety [ 22 , 23 , 24 ], teleoperation [ 25 , 26 ], human body detection and tracking [ 27 , 28 , 29 ], and gesture recognition tasks [ 30 , 31 , 32 ]. Healthcare applications involve gait analysis and elderly monitoring [ 33 , 34 , 35 , 36 ], the reconstruction of human body kinematics thanks to augmented and virtual reality software based on Kinect v2 [ 37 , 38 ].…”

Section: Introductionmentioning

confidence: 99%

Experimental Procedure for the Metrological Characterization of Time-of-Flight Cameras for Human Body 3D Measurements

Pasinetti

Nuzzi

Luchetti

et al. 2023

Sensors

View full text Add to dashboard Cite

Time-of-flight cameras are widely adopted in a variety of indoor applications ranging from industrial object measurement to human activity recognition. However, the available products may differ in terms of the quality of the acquired point cloud, and the datasheet provided by the constructors may not be enough to guide researchers in the choice of the perfect device for their application. Hence, this work details the experimental procedure to assess time-of-flight cameras’ error sources that should be considered when designing an application involving time-of-flight technology, such as the bias correction and the temperature influence on the point cloud stability. This is the first step towards a standardization of the metrological characterization procedure that could ensure the robustness and comparability of the results among tests and different devices. The procedure was conducted on Kinect Azure, Basler Blaze 101, and Basler ToF 640 cameras. Moreover, we compared the devices in the task of 3D reconstruction following a procedure involving the measure of both an object and a human upper-body-shaped mannequin. The experiment highlighted that, despite the results of the previously conducted metrological characterization, some devices showed evident difficulties in reconstructing the target objects. Thus, we proved that performing a rigorous evaluation procedure similar to the one proposed in this paper is always necessary when choosing the right device.

show abstract

3D Object Pose Estimation for Robotic Packing Applications

Cited by 7 publications

References 14 publications

Evaluation of the use of box size priors for 6D plane segment tracking from point clouds with applications in cargo packing

Evaluation of the use of box size priors for 6D plane segment tracking from point clouds with applications in cargo packing

Incremental 3D Cuboid Modeling with Drift Compensation

Experimental Procedure for the Metrological Characterization of Time-of-Flight Cameras for Human Body 3D Measurements

Contact Info

Product

Resources

About