Perspectives of RealSense and ZED Depth Sensors for Robotic Vision Applications

Tadić, Vladimir; Tóth, Attila; Vizvári, Zoltán; Klincsik, Mihaly; Sari, Zaki; Šarčević, Peter; Sárosi, József; Bíró, István

doi:10.3390/machines10030183

Cited by 32 publications

(27 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…As the accuracy of the depth map depends on the distance from the scanned object [ 21 ], we established experiments to analyze accuracy by the scanning distance. We scanned an object of known geometry and estimated the error against the ground truth.…”

Section: Experiments and Resultsmentioning

confidence: 99%

3D Reconstruction Using 3D Registration-Based ToF-Stereo Fusion

Jung

Lee

et al. 2022

Sensors

View full text Add to dashboard Cite

Depth sensing is an important issue in many applications, such as Augmented Reality (AR), eXtended Reality (XR), and Metaverse. For 3D reconstruction, a depth map can be acquired by a stereo camera and a Time-of-Flight (ToF) sensor. We used both sensors complementarily to improve the accuracy of 3D information of the data. First, we applied a generalized multi-camera calibration method that uses both color and depth information. Next, depth maps of two sensors were fused by 3D registration and reprojection approach. Then, hole-filling was applied to refine the new depth map from the ToF-stereo fused data. Finally, the surface reconstruction technique was used to generate mesh data from the ToF-stereo fused pointcloud data. The proposed procedure was implemented and tested with real-world data and compared with various algorithms to validate its efficiency.

show abstract

Section: Experiments and Resultsmentioning

confidence: 99%

3D Reconstruction Using 3D Registration-Based ToF-Stereo Fusion

Jung

Lee

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…ZED 2i is designed for outdoor applications and challenging agricultural environments (ZED Product Portfolio, 2022), which should be applied for further research in robotic vision field (Tadic et al, 2022). As a depth camera, it collects depth information based on stereoscopic technology (Sosa‐León & Schwering, 2022; Tadic et al, 2022). It is important to note that depth measuring principle of ZED 2i is distinguished from Azure Kinect and RealSense D435.…”

Section: Methodsmentioning

confidence: 99%

“…However, the depth map generated by low-cost Kinect devices has some problems, such as depth missing areas on the edge of target object, RGB images' edge do not aligned with the edge of target object, and more depth noise in depth map (Wang et al, 2016). (Tadic et al, 2022).…”

Section: Azure Kinectmentioning

confidence: 99%

Performance evaluation of newly released cameras for fruit detection and localization in complex kiwifruit orchard environments

Liu,

Jing,

Jiang

et al. 2024

Journal of Field Robotics

View full text Add to dashboard Cite

Consumer RGB‐D and binocular stereo cameras were applied to fruit detection and localization. However, few studies are documented on performance comparison of newly released cameras under same scene in complex orchard. This study evaluates performance of consumer RGB‐D and binocular stereo cameras based on YOLOv5x for kiwifruit detection and localization and selection of optimal one with better application in complex orchard environment. Firstly, Azure Kinect, RealSense D435, and ZED 2i cameras were employed to capture images of kiwifruit canopies. Subsequently, YOLOv5x was applied to train and detect kiwifruits and calyxes in the images. Meanwhile, an overlap‐partitioning detection strategy was applied on kiwifruit and calyx detection. Additionally, spatial coordinate transformation was performed by integrating camera's extrinsic parameters and depth map generated by each camera. Finally, three‐dimensional coordinates of calyxes were calculated and compared with ground truth, followed by localization accuracy of calyxes were analyzed. Results show that YOLOv5x obtained mean average precision of 93.2%, 91.3%, and 95.8% for three cameras on kiwifruit and calyx detection, respectively. Overlap‐partitioning detection strategy improved the calyx detection and significantly increased average precision by 13.00%, 16.30%, and 7.70%, respectively. The mean absolute deviation of calyx coordinates on Y‐axis is relatively high for ZED 2i at 8.44 mm in comparison of 6.67 mm for Azure Kinect, while RealSense D435 achieved minimum of 10.42 mm on X‐axis and 18.33 mm on Z‐axis. Average spatial localization speed of calyxes in one image was 0.164 s, 0.037 s, and 0.062 s for Azure Kinect, RealSense D435, and ZED 2i, respectively. These results indicate the excellent performance of RealSense D435 than Azure Kinect and ZED 2i in kiwifruit orchard, which could be a valuable reference for other orchards to select a camera with high precision localization capacity.

show abstract

“…In recent years, a number of small-sized relatively cheap sensors became available in computer vision applications, such as the Xtion [11], PrimeSense and Kinect sensors [12][13][14]. These devices are mostly used in prototyping; however, their implementation in production is also possible, as the depth sensing accuracy is feasible for small-sized mobile robots.…”

Section: Related Workmentioning

confidence: 99%

ISVD-Based Advanced Simultaneous Localization and Mapping (SLAM) Algorithm for Mobile Robots

Somlyai

Vamossy

2022

Machines

View full text Add to dashboard Cite

In the case of simultaneous localization and mapping, route planning and navigation are based on data captured by multiple sensors, including built-in cameras. Nowadays, mobile devices frequently have more than one camera with overlapping fields of view, leading to solutions where depth information can also be gathered along with ordinary RGB color data. Using these RGB-D sensors, two- and three-dimensional point clouds can be recorded from the mobile devices, which provide additional information for localization and mapping. The method of matching point clouds during the movement of the device is essential: reducing noise while having an acceptable processing time is crucial for a real-life application. In this paper, we present a novel ISVD-based method for displacement estimation, using key points detected by SURF and ORB feature detectors. The ISVD algorithm is a fitting procedure based on SVD resolution, which removes outliers from the point clouds to be fitted in several steps. The developed method removes these outlying points in several steps, in each iteration examining the relative error of the point pairs and then progressively reducing the maximum error for the next matching step. An advantage over relevant methods is that this method always gives the same result, as no random steps are included.

show abstract

Perspectives of RealSense and ZED Depth Sensors for Robotic Vision Applications

Cited by 32 publications

References 48 publications

3D Reconstruction Using 3D Registration-Based ToF-Stereo Fusion

3D Reconstruction Using 3D Registration-Based ToF-Stereo Fusion

Performance evaluation of newly released cameras for fruit detection and localization in complex kiwifruit orchard environments

ISVD-Based Advanced Simultaneous Localization and Mapping (SLAM) Algorithm for Mobile Robots

Contact Info

Product

Resources

About