Beast

Eldawy, Ahmed; Hristidis, Vagelis; Ghosh, Saheli; Saeedan, Majid; Sevim, Akil; Siddique, A.B.; Singla, Samriddhi; Sivaram, Ganesh; Vu, Tin; Zhang, Yaming

doi:10.1145/3459637.3481897

Cited by 19 publications

(4 citation statements)

References 72 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The next step of our work will primarily focus on three areas: 1) Leveraging similar projects like Beast [15] to provide data-driven spatial partitioning. 2) Developing additional features to enhance user experience and performance, such as supporting the SRID as the Geometry column's attribute, utilizing the Hilbert Curve [44] as a sorting order transformation function, implementing the FP-delta method proposed in spatial Parquet [13], and incorporating page-level filter in Parquet files.…”

Section: Discussionmentioning

confidence: 99%

“…Apache Sedona provides rich spatial functions and optimized spatial joins on this basis. There are also projects similar to Apache Sedona, such as Beast and Mosaic [15], [16], the former is decoupled from the storage layer and may require improvement in terms of user experience and performance, and the latter is currently still in development, and the richness of spatial functions is not yet sufficient.…”

Section: E Spatial Processing Solutionmentioning

confidence: 99%

See 1 more Smart Citation

GeoLake: Bringing Geospatial Support to Lakehouses

Zhang,

Peng,

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Over the past decade, geospatial data has witnessed significant expansion, coupled with swift technological advancements. However, its adoption in various industries has been partially constrained by the absence of an efficient data system that can keep pace with these advancements. Among potential solutions, the Lakehouse approach emerges as the most promising. However, existing Lakehouse systems lack integrated support for geospatial data. To address this gap, we present GeoLake, the industry's first universal solution for geospatial data tailored to Lakehouse systems. GeoLake introduces a range of pivotal features, including native support for storing geometry values in Lakehouse tables, and enables seamless querying and processing of geospatial tables using Spatial SQL. Furthermore, GeoLake incorporates advanced techniques such as spatial partitioning and spatial predicate pushdown within the Lakehouse architecture. These techniques expedite geospatial queries through efficient data skipping and optimized data layout. Additionally, GeoLake incorporates a thoughtfully designed Parquet file format that is tailored for the rapid scanning of encoded geometry values. Our experiments with real-world, large-scale geospatial datasets demonstrate the significant performance improvements achieved by GeoLake. Moreover, several industries, including automotive manufacturing, automobile insurance, and smart city initiatives, some of which are Fortune 500 enterprises, are actively adopting or considering the implementation of GeoLake to address the complexities associated with managing and analyzing geospatial data at scale.INDEX TERMS lakehouse, geospatial, spatial parquet format, spatial partitioning, spatial predicate pushdown, serializer and deserializer.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: E Spatial Processing Solutionmentioning

confidence: 99%

GeoLake: Bringing Geospatial Support to Lakehouses

Zhang,

Peng,

et al. 2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…We implement four spatial join algorithms with their original design on Beast [16] -a Spark based system for big spatial data management. Beast is deployed on a Spark 3.0 cluster of one master node with 128GB RAM and 2×8-core Intel Xeon CPU E5-2609 v4 @1.7GHz, and 12 executor nodes each with 2×6-core Intel Xeon E5-2603 @1.7GHz and 10TB HDD.…”

Section: Methodsmentioning

confidence: 99%

“…Additionally, NASA EOSDIS consistently integrates 6.4 TB of data into its repositories daily [55]. The traditional Spatial DBMS technology struggled to handle these petabytes of data, prompting the emergence of numerous significant spatial data management systems, such as SpatialHadoop [17], Hadoop-GIS [2], SparkGIS [6], Beast [16], Apache Sedona (formerly known as GeoSpark) [69], Simba [66], and many others [19]. One of the most important and challenging operations in all these systems is spatial join.…”

Section: Introductionmentioning

confidence: 99%

A learning-based framework for spatial join processing: estimation, optimization and tuning

Vu,

Belussi,

Migliorini

et al. 2024

The VLDB Journal

Self Cite

View full text Add to dashboard Cite

The importance and complexity of spatial join operation resulted in the availability of many join algorithms, some of which are tailored for big-data platforms like Hadoop and Spark. The choice among them is not trivial and depends on different factors. This paper proposes the first machine-learning-based framework for spatial join query optimization which can accommodate both the characteristics of spatial datasets and the complexity of the different algorithms. The main challenge is how to develop portable cost models that once trained can be applied to any pair of input datasets, because they are able to extract the important input characteristics, such as data distribution and spatial partitioning, the logic of spatial join algorithms, and the relationship between the two input datasets. The proposed system defines a set of features that can be computed efficiently for the data to catch the intricate aspects of spatial join. Then, it uses these features to train five machine learning models that are used to identify the best spatial join algorithm. The first two are regression models that estimate two important measures of the spatial join performance and they act as the cost model. The third model chooses the best partitioning strategy to use with spatial join. The fourth and fifth models further tune two important parameters, number of partitions and plane-sweep direction, to get the best performance. Experiments on large-scale synthetic and real data show the efficiency of the proposed models over baseline methods.

show abstract

TRindex: Distributed Double-Layer Road Network Trajectory Index

Chen

Tang

et al. 2023

Communications in Computer and Information Science

View full text Add to dashboard Cite

Beast

Cited by 19 publications

References 72 publications

GeoLake: Bringing Geospatial Support to Lakehouses

GeoLake: Bringing Geospatial Support to Lakehouses

A learning-based framework for spatial join processing: estimation, optimization and tuning

TRindex: Distributed Double-Layer Road Network Trajectory Index

Contact Info

Product

Resources

About