Chisel: Real Time Large Scale 3D Reconstruction Onboard a Mobile Device using Spatially Hashed Signed Distance Fields

Klingensmith, Matthew; Dryanovski, Ivan; Srinivasa, Siddhartha S.; Xiao, Jizhong

doi:10.15607/rss.2015.xi.040

Cited by 150 publications

(137 citation statements)

References 34 publications

Supporting

Mentioning

133

Contrasting

Unclassified

Order By: Relevance

“…The raw point clouds used in the present study disregard non-surface information and can be susceptible to sensor noise. Future work may therefore seek to improve the quality of 3D reconstruction by experimenting with alternative techniques such as optimised variants of occupancy grid mapping and truncated signed distance fields (TSDF) which use the passthrough data of emanating rays to provide more detailed volumetric information [82,83]. Ideally, the performance of motion tracking and depth sensing would also be tested with a wider range of environments, vegetation types, and movement speeds to closer emulate conditions found in more challenging field deployments.…”

Section: Discussionmentioning

confidence: 99%

Joining the dots: reconstructing 3D environments and movement paths using animal-borne devices

McClune

2018

Anim Biotelemetry

View full text Add to dashboard Cite

Background: Animal-attached sensors are increasingly used to provide insights on behaviour and physiology. However, such tags usually lack information on the structure of the surrounding environment from the perspective of a study animal and thus may be unable to identify potentially important drivers of behaviour. Recent advances in robotics and computer vision have led to the availability of integrated depth-sensing and motion-tracking mobile devices. These enable the construction of detailed 3D models of an environment within which motion can be tracked without reliance on GPS. The potential of such techniques has yet to be explored in the field of animal biotelemetry. This report trials an animal-attached structured light depth-sensing and visual-inertial odometry motion-tracking device in an outdoor environment (coniferous forest) using the domestic dog (Canis familiaris) as a compliant test species.Results: A 3D model of the forest environment surrounding the subject animal was successfully constructed using point clouds. The forest floor was labelled using a progressive morphological filter. Trees trunks were modelled as cylinders and identified by random sample consensus. The predicted and actual presence of trees matched closely, with an object-level accuracy of 93.3%. Individual points were labelled as belonging to tree trunks with a precision, recall, and F β score of 1.00, 0.88, and 0.93, respectively. In addition, ground-truth tree trunk radius measurements were not significantly different from random sample consensus model coefficient-derived values. A first-person view of the 3D model was created, illustrating the coupling of both animal movement and environment reconstruction. Conclusions:Using data collected from an animal-borne device, the present study demonstrates how terrain and objects (in this case, tree trunks) surrounding a subject can be identified by model segmentation. The device pose (position and orientation) also enabled recreation of the animal's movement path within the 3D model. Although some challenges such as device form factor, validation in a wider range of environments, and direct sunlight interference remain before routine field deployment can take place, animal-borne depth sensing and visual-inertial odometry have great potential as visual biologging techniques to provide new insights on how terrestrial animals interact with their environments.

show abstract

Section: Discussionmentioning

confidence: 99%

Joining the dots: reconstructing 3D environments and movement paths using animal-borne devices

McClune

2018

Anim Biotelemetry

View full text Add to dashboard Cite

show abstract

“…We fuse all depth images obtained at different camera poses into a global dense map using an uncertainty‐aware truncated signed distance field (TSDF) fusion approach. Our method is developed from the open source CHISEL TSDF implementation . Improvements include uncertainty‐aware depth fusion (Section ) and algorithm parallelization (Section ).…”

Section: Monocular Dense Mappingmentioning

confidence: 99%

Autonomous aerial navigation using monocular visual‐inertial fusion

Lin

Gao

Qin

et al. 2017

Journal of Field Robotics

197

166

View full text Add to dashboard Cite

Autonomous micro aerial vehicles (MAVs) have cost and mobility benefits, making them ideal robotic platforms for applications including aerial photography, surveillance, and search and rescue. As the platform scales down, MAVs become more capable of operating in confined environments, but it also introduces significant size and payload constraints. A monocular visual-inertial navigation system (VINS), consisting only of an inertial measurement unit (IMU) and a camera, becomes the most suitable sensor suite in this case, thanks to its light weight and small footprint.In fact, it is the minimum sensor suite allowing autonomous flight with sufficient environmental awareness. In this paper, we show that it is possible to achieve reliable online autonomous navigation using monocular VINS. Our system is built on a customized quadrotor testbed equipped with a fisheye camera, a low-cost IMU, and heterogeneous onboard computing resources. The backbone of our system is a highly accurate optimization-based monocular visual-inertial state estimator with online initialization and self-extrinsic calibration. An onboard GPU-based monocular dense mapping module that conditions on the estimated pose provides wide-angle situational awareness. Finally, an online trajectory planner that operates directly on the incrementally built threedimensional map guarantees safe navigation through cluttered environments. Extensive experimental results are provided to validate individual system modules as well as the overall performance in both indoor and outdoor environments.

show abstract

“…The sparse maps are aligned and refined using visual keypoint based loop closure [24] and batch optimization. The dense 3D reconstruction is based on the Google Tango framework as well, which is closely related to its OpenSource version OpenChisel [25].…”

Section: A Setupmentioning

confidence: 99%

TSDF-based change detection for consistent long-term dense reconstruction and dynamic object discovery

Fehr

Furrer

Dryanovski

et al. 2017

2017 IEEE International Conference on Robotics and Automation (ICRA)

Self Cite

View full text Add to dashboard Cite

Abstract-Robots that are operating for extended periods of time need to be able to deal with changes in their environment and represent them adequately in their maps. In this paper, we present a novel 3D reconstruction algorithm based on an extended Truncated Signed Distance Function (TSDF) that enables to continuously refine the static map while simultaneously obtaining 3D reconstructions of dynamic objects in the scene. This is a challenging problem because map updates happen incrementally and are often incomplete. Previous work typically performs change detection on point clouds, surfels or maps, which are not able to distinguish between unexplored and empty space. In contrast, our TSDF-based representation naturally contains this information and thus allows us to more robustly solve the scene differencing problem. We demonstrate the algorithms performance as part of a system for unsupervised object discovery and class recognition. We evaluated our algorithm on challenging datasets that we recorded over several days with RGB-D enabled tablets. To stimulate further research in this area, all of our datasets are publicly available 3 .

show abstract

Chisel: Real Time Large Scale 3D Reconstruction Onboard a Mobile Device using Spatially Hashed Signed Distance Fields

Cited by 150 publications

References 34 publications

Joining the dots: reconstructing 3D environments and movement paths using animal-borne devices

Joining the dots: reconstructing 3D environments and movement paths using animal-borne devices

Autonomous aerial navigation using monocular visual‐inertial fusion

TSDF-based change detection for consistent long-term dense reconstruction and dynamic object discovery

Contact Info

Product

Resources

About