Massive point cloud data management: Design, implementation and execution of a point cloud benchmark

Oosterom, Peter van; Martinez-Rubi, O.; Ivanova, Milena; Horhammer, Mike; Geringer, Daniel; Ravada, Siva; Tijssen, T.P.M.; Kodde, Martin; Gonçalves, Romulo

doi:10.1016/j.cag.2015.01.007

Cited by 82 publications

(94 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent work [1,12,29] illustrates the potential of column-store DBMSs to meet point cloud management requirements. The MonetDB demo [1] showcases the declarative power of DBMS when managing point cloud data that is enriched with semantics from different sources.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Dictionary Compression in Point Cloud Data Management

Pavlović

Bastian

Gildhoff

et al. 2017

Proceedings of the 25th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

View full text Add to dashboard Cite

Nowadays, massive amounts of point cloud data can be collected thanks to advances in data acquisition and processing technologies like dense image matching and airborne LiDAR (Light Detection and Ranging) scanning. With the increase in volume and precision, point cloud data offers a useful source of information for natural resource management, urban planning, self-driving cars and more. At the same time, the scale at which point cloud data is produced, introduces management challenges: it is important to achieve efficiency both in terms of querying performance and space requirements. Traditional file-based solutions to point cloud management offer space efficiency, however, cannot scale to such massive data and provide the same declarative power as a database management system (DBMS).In this paper, we propose a time-and space-efficient solution to storing and managing point cloud data in main memory columnstore DBMS. Our solution, Space-Filling Curve Dictionary-Based Compression (SFC-DBC), employs dictionary-based compression in the spatial data management domain and enhances it with indexing capabilities by using space-filling curves. It does so by constructing the space-filling curve over a compressed, artificially introduced 3D dictionary space. Consequently, SFC-DBC significantly optimizes query execution, and yet it does not require additional storage resources, compared to traditional dictionary-based compression. With respect to space-filling curve-based approaches, it minimizes storage footprint and increases resilience to skew. As a proof of concept, we develop and evaluate our approach as a research prototype in the context of SAP HANA. SFC-DBC outperforms other dictionary-based compression schemes by up to 61% in terms of space and up to 9.4x in terms of query performance.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Therefore, they require additional space resources which can introduce significant overhead, particularly for solutions based on the flat storage model [29]. An exception is the previously introduced MortonreplacedXY approach [15].…”

Section: Related Workmentioning

confidence: 99%

Dictionary Compression in Point Cloud Data Management

Pavlović

Bastian

Gildhoff

et al. 2017

Proceedings of the 25th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

View full text Add to dashboard Cite

show abstract

“…It is already a large amount of files to be inspected for a simple selection by a robust file-based solution like Rapidlasso LAStools [5]. In [18], the authors for LAStools had to use a DBMS to store the metadata of each file in order to avoid the inspection of each file header, and run a lassort and lasindex to boost query performance. Such ETL process had the same cost as the data loading cost of a DBMS.…”

Section: File-based Solutionsmentioning

confidence: 99%

“…Such spatial order is in most of the cases not exploited by the used compression techniques. An exception is PostgreSQL which uses compression techniques based on the spatial continuity of the blocks to offer better storage [18]. The same principles are followed by LAStools through lassort and lasindex.…”

Section: Indexing Point Cloud Datamentioning

confidence: 99%

GIS navigation boosted by column stores

Alvanaki

Gonçalves

Ivanova³

et al. 2015

Proc. VLDB Endow.

Self Cite

View full text Add to dashboard Cite

Earth observation sciences, astronomy, and seismology have large data sets which have inherently rich spatial and geospatial information. In combination with large collections of semantically rich objects which have a large number of thematic properties, they form a new source of knowledge for urban planning, smart cities and natural resource management.Modeling and storing these properties indicating the relationships between them is best handled in a relational database. Furthermore, the scalability requirements posed by the latest 26-attribute light detection and ranging (LI-DAR) data sets are a challenge for file-based solutions.In this demo we show how to query a 640 billion point data set using a column store enriched with GIS functionality. Through a lightweight and cache conscious secondary index called Imprints, spatial queries performance on a flat table storage is comparable to traditional file-based solutions. All the results are visualised in real time using QGIS.

show abstract

“…SFCs have the ability to cluster points close in reality, close on the curve. The improved organisation is introduced in van Oosterom et al (2015) and extended in Martinez-Rubi et al (2015) for the two spatial dimensions.…”

Section: Introductionmentioning

confidence: 99%

Using a Space Filling Curve Approach for the Management Of Dynamic Point Clouds

Psomadaki¹,

Oosterom²,

Tijssen³

et al. 2016

ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

S.Psomadaki@student.tudelft.nl, (P.J.M.vanOosterom, T.P.M.Tijssen)@tudelft.nl b Deltares, 2600 MH, Delft, the Netherlands -Fedor.Baart@deltares.nl KEY WORDS: Point cloud data, Space filling curve, Spatio-temporal data, Benchmark, DBMS ABSTRACT:Point cloud usage has increased over the years. The development of low-cost sensors makes it now possible to acquire frequent point cloud measurements on a short time period (day, hour, second). Based on the requirements coming from the coastal monitoring domain, we have developed, implemented and benchmarked a spatio-temporal point cloud data management solution. For this reason, we make use of the flat model approach (one point per row) in an Index Organised Table within a RDBMS and an improved spatio-temporal organisation using a Space Filling Curve approach. Two variants coming from two extremes of the space -time continuum are also taken into account, along with two treatments of the z dimension: as attribute or as part of the space filling curve. Through executing a benchmark we elaborate on the performance -loading and querying time-, and storage required by those different approaches. Finally, we validate the correctness and suitability of our method, through an out-of-the-box way of managing dynamic point clouds.

show abstract

Massive point cloud data management: Design, implementation and execution of a point cloud benchmark

Cited by 82 publications

References 11 publications

Dictionary Compression in Point Cloud Data Management

Dictionary Compression in Point Cloud Data Management

GIS navigation boosted by column stores

Using a Space Filling Curve Approach for the Management Of Dynamic Point Clouds

Contact Info

Product

Resources

About