Are updatable learned indexes ready?

Wongkham, Chaichon; Lu, Baotong; Liu, Chris; Zhong, Zhicong; Lo, Eric; Wang, Tianzheng

doi:10.14778/3551793.3551848

Cited by 19 publications

(7 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Kipf et al used real-world datasets for benchmarks for learned indexes but only focused on search performance over sorted datasets [37]. Simultaneously to our work, skewness of a dataset is similarly analyzed for learned indexes, where skewness is called "hardness" [60].…”

Section: Related Workmentioning

confidence: 97%

DyTIS: A Dynamic Dataset Targeted Index Structure Simultaneously Efficient for Search, Insert, and Scan

Yang

Yoon

Yun

et al. 2023

Proceedings of the Eighteenth European Conference on Computer Systems

View full text Add to dashboard Cite

Many datasets in real life are complex and dynamic, that is, their key densities are varied over the whole key space and their key distributions change over time. It is challenging for an index structure to efficiently support all key operations for data management, in particular, search, insert, and scan, for such dynamic datasets. In this paper, we present DyTIS (Dynamic dataset Targeted Index Structure), an index that targets dynamic datasets. DyTIS, though based on the structure of Extendible hashing, leverages the CDF of the key distribution of a dataset, and learns and adjusts its structure as the dataset grows. The key novelty behind DyTIS is to group keys by the natural key order and maintain keys in sorted order in each bucket to support scan operations within a hash index. We also define what we refer to as a dynamic dataset and propose a means to quantify its dynamic characteristics. Our experimental results show that DyTIS provides higher performance than the state-of-the-art learned index for the dynamic datasets considered.

show abstract

Section: Related Workmentioning

confidence: 97%

DyTIS: A Dynamic Dataset Targeted Index Structure Simultaneously Efficient for Search, Insert, and Scan

Yang

Yoon

Yun

et al. 2023

Proceedings of the Eighteenth European Conference on Computer Systems

View full text Add to dashboard Cite

show abstract

“…In this case, the time complexity for lookup is O(1), and the memory usage of the model is usually smaller than that of the traditional index structure. This is why learned index has become popular these days [10,11].…”

Section: Learned Indexmentioning

confidence: 99%

“…Typical examples include B+-tree [5], trie [6], radix tree [7], ordered index [8], and so on. Recently, a new approach, called learned index, has gained attention [9][10][11]. The fundamental idea of learned index is making use of machine learning algorithms for search.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Empirical Study of Segmented Linear Regression Search in LevelDB

et al. 2023

View full text Add to dashboard Cite

The purpose of this paper is proposing a novel search mechanism, called SLR (Segmented Linear Regression) search, based on the concept of learned index. It is motivated by our observation that a lot of big data, collected and used by previous studies, have a linearity property, meaning that keys and their stored locations show a strong linear correlation. This observation leads us to design SLR search where we apply segmentation into the well-known machine learning algorithm, linear regression, for identifying a location from a given key. We devise two segmentation techniques, equal-size and error-aware, with the consideration of both prediction accuracy and segmentation overhead. We implement our proposal in LevelDB, Google’s key-value store, and verify that it can improve search performance by up to 12.7%. In addition, we find that the equal-size technique provides efficiency in training while the error-aware one is tolerable to noisy data.

show abstract

“…Despite the progress of updatable learned indexes, they do not always beat traditional indexes in mixed read-and-write scenarios. Wongkham et al tested ALEX, LIPP, and an updatable version of the PGM in multiple scenarios in their benchmark (Wongkham et al, 2022). Among their conclusions is that space efficiency is not a guarantee of updatable learned indexes.…”

Section: Introductionmentioning

confidence: 99%

Using Learned Indexes to Improve Time Series Indexing Performance on Embedded Sensor Devices

Ding¹,

Carvalho²,

Lawrence³

2023

Preprint

View full text Add to dashboard Cite

Efficiently querying data on embedded sensor and IoT devices is challenging given the very limited memory and CPU resources. With the increasing volumes of collected data, it is critical to process, filter, and manipulate data on the edge devices where it is collected to improve efficiency and reduce network transmissions. Existing embedded index structures do not adapt to the data distribution and characteristics. This paper demonstrates how applying learned indexes that develop space efficient summaries of the data can dramatically improve the query performance and predictability. Learned indexes based on linear approximations can reduce the query I/O by 50 to 90% and improve query throughput by a factor of 2 to 5, while only requiring a few kilobytes of RAM. Experimental results on a variety of time series data sets demonstrate the advantages of learned indexes that considerably improve over the state-of-the-art index algorithms.

show abstract

Are updatable learned indexes ready?

Cited by 19 publications

References 29 publications

DyTIS: A Dynamic Dataset Targeted Index Structure Simultaneously Efficient for Search, Insert, and Scan

DyTIS: A Dynamic Dataset Targeted Index Structure Simultaneously Efficient for Search, Insert, and Scan

An Empirical Study of Segmented Linear Regression Search in LevelDB

Using Learned Indexes to Improve Time Series Indexing Performance on Embedded Sensor Devices

Contact Info

Product

Resources

About