ALEX: An Updatable Adaptive Learned Index

Ding, Jialin; Minhas, Umar Farooq; Yu, Jia; Wang, Chi; Do, Jaeyoung; Li, Yinan; Zhang, Hantian; Chandramouli, Badrish; Gehrke, Johannes; Kossmann, Donald; Lomet, David; Kraska, Tim

doi:10.1145/3318464.3389711

Cited by 172 publications

(125 citation statements)

References 17 publications

Supporting

Mentioning

122

Contrasting

Order By: Relevance

“…Recursive model indexes (RMIs) are one such class of models [8] (although others [3][4][5]11] exist as well) , combining simpler machine learning models together into a multistaged structure. For example, as depicted in Figure 1, an RMI with two stages, a linear stage and a cubic stage, would first use a linear model to make an initial prediction of an index for a specific key (stage 1).…”

Section: Introductionmentioning

confidence: 99%

CDFShop: Exploring and Optimizing Learned Index Structures

Marcus

Zhang

Kraska

2020

Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data

Self Cite

View full text Add to dashboard Cite

Indexes are a critical component of data management applications. While tree-like structures (e.g., B-Trees) have been employed to great success, recent work suggests that index structures powered by machine learning models (learned index structures) can achieve low lookup times with a reduced memory footprint. This demonstration showcases CDFShop, a tool to explore and optimize recursive model indexes (RMIs), a type of learned index structure. This demonstration allows audience members to (1) gain an intuition about various tuning parameters of RMIs and why learned index structures can greatly accelerate search, and (2) understand how automatic optimization techniques can be used to explore space/time tradeoffs within the space of RMIs.

show abstract

Section: Introductionmentioning

confidence: 99%

CDFShop: Exploring and Optimizing Learned Index Structures

Marcus

Zhang

Kraska

2020

Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data

Self Cite

View full text Add to dashboard Cite

show abstract

“…Closest to our work is a proposal of learned partition adviser using deep RL [19]; it focuses on replication and coarse-grained partitioning (e.g., hash) along entire attribute(s), unlike qd-tree which partitions based on a rich set of fine-grained candidate cuts. In this space, machine learning has also been used to revisit tuning [47], workload forecasting [28], data structures and indexes [12,20,23,32], and query optimization [13,24,29,50]. Our qd-tree may be viewed as a learned physical design or indexing tool: it optimizes for scan-based workloads, common in big data analytics, to minimize the I/O cost of block accesses.…”

Section: Related Workmentioning

confidence: 99%

Qd-tree: Learning Data Layouts for Big Data Analytics

Yang

Chandramouli

Wang

et al. 2020

Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data

Self Cite

View full text Add to dashboard Cite

Corporations today collect data at an unprecedented and accelerating scale, making the need to run queries on large datasets increasingly important. Technologies such as columnar block-based data organization and compression have become standard practice in most commercial database systems. However, the problem of best assigning records to data blocks on storage is still open. For example, today's systems usually partition data by arrival time into row groups, or range/hash partition the data based on selected fields. For a given workload, however, such techniques are unable to optimize for the important metric of the number of blocks accessed by a query. This metric directly relates to the I/O cost, and therefore performance, of most analytical queries. Further, they are unable to exploit additional available storage to drive this metric down further. In this paper, we propose a new framework called a querydata routing tree, or qd-tree, to address this problem, and propose two algorithms for their construction based on greedy and deep reinforcement learning techniques. Experiments over benchmark and real workloads show that a qd-tree can provide physical speedups of more than an order of magnitude compared to current blocking schemes, and can reach within 2× of the lower bound for data skipping based on selectivity, while providing complete semantic descriptions of created blocks.

show abstract

“…If we follow this assumption, by learning the Cumulative Distribution Function (CDF) of the input data, the mapping function of an index can be learned. Due to the complexity of the CDF, a single ML model learned over the complete data cannot provide the desired accuracy [4]. To address this issue, a Recursive Model Index (RMI, for short) has been introduced.…”

Section: Part 1: Learned Index Structuresmentioning

confidence: 99%

“…These initial works have been focused on read-only workloads. To deal with updates, a new class of updatable adaptive learned indexes has been proposed, e.g., [4]. It has been demonstrated that a careful space-time trade-off can lead to an updatable data structure.…”

Section: Introductionmentioning

confidence: 99%

A Tutorial on Learned Multi-dimensional Indexes

Al‐Mamun

Aref

2020

Proceedings of the 28th International Conference on Advances in Geographic Information Systems

View full text Add to dashboard Cite

Recently, Machine Learning (ML, for short) has been successfully applied to database indexing. Initial experimentation on Learned Indexes has demonstrated better search performance and lower space requirements than their traditional database counterparts. Numerous attempts have been explored to extend learned indexes to the multi-dimensional space. This makes learned indexes potentially suitable for spatial databases. The goal of this tutorial is to provide up-to-date coverage of learned indexes both in the single and multi-dimensional spaces. The tutorial covers over 25 learned indexes. The tutorial navigates through the space of learned indexes through a taxonomy that helps classify the covered learned indexes both in the single and multi-dimensional spaces. CCS CONCEPTS • Database Systems → Indexing; • Machine Learning → ML for Systems.

show abstract

ALEX: An Updatable Adaptive Learned Index

Cited by 172 publications

References 17 publications

CDFShop: Exploring and Optimizing Learned Index Structures

CDFShop: Exploring and Optimizing Learned Index Structures

Qd-tree: Learning Data Layouts for Big Data Analytics

A Tutorial on Learned Multi-dimensional Indexes

Contact Info

Product

Resources

About