Object-aware guidance for autonomous scene reconstruction

Liu, Ligang; Xia, Xi; Sun, Han Xu; Shen, Qi; Xu, Juzhan; Chen, Bin; Huang, Hui; Xu, Kai

doi:10.1145/3197517.3201295

Cited by 24 publications

(21 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ye et al [YLL∗18] propose navigation policy learning guided by active object detection and recognition. The work in [LXS∗18] is the most similar in spirit to ours. They develop a data‐driven solution to autonomous object detection and recognition with one navigation pass in an indoor room.…”

Section: Related Workmentioning

confidence: 78%

“…Online scene understanding is a different paradigm in which acquisition and analysis are intertwined [XHS∗15, LXS∗18, YLL∗18]: while scene analysis is conducted online based on the progressively acquired scene data, scene scanning, on the other hand, is driven by the requirement of efficient scene understanding. Such a coupled solution fits well for robot‐operated autonomous scene understanding.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Active Scene Understanding via Online Semantic Reconstruction

Zheng

Zhu

Zhang

et al. 2019

Computer Graphics Forum

Self Cite

View full text Add to dashboard Cite

We propose a novel approach to robot‐operated active understanding of unknown indoor scenes, based on online RGBD reconstruction with semantic segmentation. In our method, the exploratory robot scanning is both driven by and targeting at the recognition and segmentation of semantic objects from the scene. Our algorithm is built on top of a volumetric depth fusion framework and performs real‐time voxel‐based semantic labeling over the online reconstructed volume. The robot is guided by an online estimated discrete viewing score field (VSF) parameterized over the 3D space of 2D location and azimuth rotation. VSF stores for each grid the score of the corresponding view, which measures how much it reduces the uncertainty (entropy) of both geometric reconstruction and semantic labeling. Based on VSF, we select the next best views (NBV) as the target for each time step. We then jointly optimize the traverse path and camera trajectory between two adjacent NBVs, through maximizing the integral viewing score (information gain) along path and trajectory. Through extensive evaluation, we show that our method achieves efficient and accurate online scene parsing during exploratory scanning.

show abstract

Section: Related Workmentioning

confidence: 78%

Section: Introductionmentioning

confidence: 99%

Active Scene Understanding via Online Semantic Reconstruction

Zheng

Zhu

Zhang

et al. 2019

Computer Graphics Forum

Self Cite

View full text Add to dashboard Cite

show abstract

“…In order to achieve an automatic acquisition and reconstruction scheme, these modern reconstruction algorithms are also required to be improved for cooperating with modern motion planning strategies [9], [10], since their performance may decline significantly when deployed for vehicle scanning instead of hand-held scanning. This is essentially due to a reduced degree-of-freedom for camera motion that restricts the RGBD sensor from continuously focusing on regions with sufficient geometric details, i.e., the actions of robots are not as flexible as humans for staying at good shooting views containing sufficient registration hints for localization and mapping.…”

Section: Introductionmentioning

confidence: 99%

HeteroFusion: Dense Scene Reconstruction Integrating Multi-Sensors

Yang

Liu

et al. 2020

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

We present a novel approach to integrate data from multiple sensor types for dense 3D reconstruction of indoor scenes in realtime. Existing algorithms are mainly based on a single RGBD camera and thus require continuous scanning of areas with sufficient geometric features. Otherwise, tracking may fail due to unreliable frame registration. Inspired by the fact that the fusion of multiple sensors can combine their strengths towards a more robust and accurate self-localization, we incorporate multiple types of sensors which are prevalent in modern robot systems, including a 2D range sensor, an inertial measurement unit (IMU), and wheel encoders. We fuse their measurements to reinforce the tracking process and to eventually obtain better 3D reconstructions. Specifically, we develop a 2D truncated signed distance field (TSDF) volume representation for the integration and ray-casting of laser frames, leading to a unified cost function in the pose estimation stage. For validation of the estimated poses in the loop-closure optimization process, we train a classifier for the features extracted from heterogeneous sensors during the registration progress. To evaluate our method on challenging use case scenarios, we assembled a scanning platform prototype to acquire real-world scans. We further simulated synthetic scans based on high-fidelity synthetic scenes for quantitative evaluation. Extensive experimental evaluation on these two types of scans demonstrate that our system is capable of robustly acquiring dense 3D reconstructions and outperforms state-of-the-art RGBD and LiDAR systems.

show abstract

“…Some scholars have also considered the optimization of the sensor’s pose during the exploration. 22 –24 The classic frontier-based approach 11,14,25 can achieve a full coverage observation of the environment by selecting the nearest accessible and unvisited frontier, but it is a one-step greedy method and cannot guarantee that the final exploration trajectory is optimal. The exploration efficiency may decrease significantly due to frequent retrace the route and the fragmentation of the unexplored region.…”

Section: Introductionmentioning

confidence: 99%

A high-efficiency, information-based exploration path planning method for active simultaneous localization and mapping

Yang

Wang

et al. 2020

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

The efficiency of exploration in an unknown scene and full coverage of the scene are essential for a robot to complete simultaneous localization and mapping actively. However, it is challenging for a robot to explore an unknown environment with high efficiency and full coverage autonomously. In this article, we propose a novel exploration path planning method based on information entropy. An information entropy map is first constructed, and its boundary features are extracted. Then a Dijkstra-based algorithm is applied to generate candidate exploration paths based on the boundary features. The dead-reckoning algorithm is used to predict the uncertainty of the robot’s pose along each candidate path. The exploration path is selected based on exploration efficiency and/or high coverage. Simulations and experiments are conducted to evaluate the proposed method’s effectiveness. The results demonstrated that the proposed method achieved not only higher exploration efficiency but also a larger coverage area.

show abstract

Object-aware guidance for autonomous scene reconstruction

Cited by 24 publications

References 29 publications

Active Scene Understanding via Online Semantic Reconstruction

Active Scene Understanding via Online Semantic Reconstruction

HeteroFusion: Dense Scene Reconstruction Integrating Multi-Sensors

A high-efficiency, information-based exploration path planning method for active simultaneous localization and mapping

Contact Info

Product

Resources

About