RM3D: Robust Data-Efficient 3D Scene Parsing via Traditional and Learnt 3D Descriptors-Based Semantic Region Merging

Abstract

Existing state-of-the-art 3D point cloud understanding methods merely perform well in a fully supervised manner. To the best of our knowledge, there exists no unified framework that simultaneously solves the downstream high-level understanding tasks including both segmentation and detection, especially when labels are extremely limited. This work presents a general and simple framework to tackle point cloud understanding when labels are limited. The first contribution is that we have done extensive methodology comparisons of traditional and learned 3D descriptors for the task of weakly supervised 3D scene understanding, and validated that our adapted traditional PFH-based 3D descriptors show excellent generalization ability across different domains. The second contribution is that we proposed a learning-based region merging strategy based on the affinity provided by both the traditional/learned 3D descriptors and learned semantics. The merging process takes both low-level geometric and high-level semantic feature correlations into consideration. Experimental results demonstrate that our framework has the best performance among the three most important weakly supervised point clouds understanding tasks including semantic segmentation, instance segmentation, and object detection even when very limited number of points are labeled. Our method, termed Region Merging 3D (RM3D), has superior performance on Scan-Net data-efficient learning online benchmarks and other four large-scale 3D understanding benchmarks under various experimental settings, outperforming current arts by a margin for various 3D understanding tasks without complicated learning strategies such as active learning.

An Integrated Visual System for Unmanned Aerial Vehicles Following Ground Vehicles: Simulations and Experiments

“…Finally, we have integrated our proposed approach with other robotics modules such as SLAM and motion/task planning as a whole system to perform UAV tracking and landing in real applications. UAV Vision is important [34], [35]. Our integrated visual system is very important for future UAV-based detection and tracking applications such as autonomous landing on moving vehicles.…”

Industrial UAV-Based Unsupervised Domain Adaptive Crack Recognitions: From Database Towards Real-Site Infrastructural Inspections

“…Various visual sensors and Light Detection And Ranging Sensors (LiDAR) sensors have been widely deployed in various robotics platforms such as automatically navigated aerial robots and ground robots [16], [17], [18], [19], [20], [21]. For example, the SLAM is widely adopted for robot localization, mapping, and navigation applications [11], [22], [23], [12], [24], [25], [26], [27], [28], [29], [30], [31], [22].…”

Construction and performance evaluation of an efficient local feature descriptor based on polar coordinates

“…On the other hand, LRA provides information exclusively along radial and elevation directions. Despite LRF's capability to furnish information for the entire local space, its repeatability along the x and y axes notably lags behind that along the Z-axis [12] , resulting in an overall repeatability markedly inferior to that of LRA. Spatial information encoded through the LRA-based approach attains heightened precision when compared to the LRF-centric methodology.…”