Mithril

Yang, Juncheng; Karimi, Reza; Saemundsson, Trausti; Wildani, Avani; Vigfússon, Ýmir

doi:10.1145/3127479.3131210

Cited by 13 publications

(5 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this chapter, we evaluate the proposed cache‐transaction‐based feature extraction method and the data group prefetching algorithm. We compare the proposed model with popular feature extraction methods and prefetching algorithms, including prefetching models based on RDF and OF, such as ANN, 28 and Mithril 20 …”

Section: Discussionmentioning

confidence: 99%

“…Another way to reduce computational complexity is to set thresholds based on data access characteristics. For example, researchers proposed Mithril 20 for mining the access relation of data with large block address distances, based on the idea that most related data have similar access times 29,30 . Mithril mines the access relations among the data with the same access times.…”

Section: Background and Motivationmentioning

confidence: 99%

“…For the data correlation mining model, the existing studies are divided into two classes. The first includes online algorithms that determine which data to prefetch when an access request arrives, such as Mithril 20 and Tombolo 34…”

Section: Related Studiesmentioning

confidence: 99%

“…Data possess fewer valuable features for correlation mining. Most existing data correlation analysis methods use the block address and the access interval between data as features and estimate data correlations by defining the distance between data 20–23 . These correlation analysis methods determine the data features from related data, and such features are called relationship defined features (RDFs).…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A data grouping model based on cache transaction for unstructured data storage systems

Zhu

Sun

et al. 2021

Int J of Intelligent Sys

View full text Add to dashboard Cite

Cache prefetching technology has become the mainstream data access optimization strategy in the Industrial Intelligent Systems (IIS) and the data centers. However, the rapidly increasing of unstructured data generates massive pairwise access relationships. Therefore, researchers have to make a choice between spatial locality and temporal locality to ensure an acceptable computational complexity. We propose cache-transaction-based data grouping model (CTDGM) to solve the problems described above by optimizing the feature representation method and grouping efficiency. First, we provide the definition of the cache transaction and propose the method for extracting the cache transaction feature (CTF). Second, we design a data chunking algorithm based on CTF and spatiotemporal locality to optimize the relationship calculation efficiency.Third, we propose CTDGM by constructing a relation graph that groups data into independent groups according to the strength of the data access relation. Based on the results of the experiment, compared with the state-of-the-art and traditional methods, our algorithm achieves an average increase in the cache hit rate of 5%-20% on the MSR, VDI-LUN, and KC

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Background and Motivationmentioning

confidence: 99%

Section: Related Studiesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A data grouping model based on cache transaction for unstructured data storage systems

Zhu

Sun

et al. 2021

Int J of Intelligent Sys

View full text Add to dashboard Cite

show abstract

“…Luckily, the data access patterns of these systems show a repetitive and predictable pattern [148], [149] providing an opportunity for performance improvements via effective caching. These observations led to the design and implementation of in-memory processing frameworks such as Spark [48] and caching/prefetching solutions [79], [89], [91]- [93], [119], [122], [123], [150], [151].…”

Section: Related Workmentioning

confidence: 99%

Informed optimization of cloud applications

Abdi

View full text Add to dashboard Cite

Cloud applications running in private data centers or public clouds are critical to every aspect of modern society, from entertainment to healthcare to financial services. The rapid growth of cloud applications is because organizations extensively leverage the general-purpose services offered by cloud platforms. Examples are general-purpose storage services such as AWS S3 [4]. These general-purpose services promise to provide high-performance and low-cost data analysis, elastic and scalable resources, and robust and reliable computations. Yet, cloud applications confront challenges to achieving these promises for several reasons: the exponential growth of data, the network limitations, the overhead of accessing the remote storage service, and the scarcity of complete information to make optimized decisions at different layers of the applications.This dissertation focuses on proposing systems to address the challenges mentioned above.We describe two systems that expand the general-purpose service APIs to exchange information critical to optimizing decisions. We also introduce components and algorithms that interpret the exchanged information and make optimized decisions.First, we describe Kariz to address the challenge of providing high-performance and scalable analytic applications with disaggregated storage-compute deployments. Kariz targets cloud applications that leverage general-purpose services such as analytical frameworks and object storage services. Kariz introduces a distributed caching system that extends the general-purpose object storage service and caches and prefetches input data accessed by the analytical framework. It extends the analytic framework APIs to exchange job dependency graphs (DAGs) and scheduling information with the cache, makes I/O scheduling decisions according to the scheduling information, estimates the run-time improvements, and predicts bandwidth to object storage service. Kariz integrates a cross-DAG scheduler and prefers pre-fetching input data shared by multiple DAGs. To build and evaluate Kariz, we characterize the job dependency graphs(DAGs) generated by analytic frameworks in a workload from in-production clusters.Then, we describe the elements of Kariz: a straggler-resistant partial cache system, a cache controller that run alongside the analytic frameworks-i.e. Spark and Pig-, and two graph-aware caching and pre-fetching algorithm for optimizing the caching of individual queries and three policies for optimizing across queries from, potentially, multiple different clusters. With an vi algorithm that fully exploits the rich information available from Kariz, we demonstrate major speedups (as much as 3×) for TPC-H and TPC-DS.Second, we describe Palette, to address the growing challenge of having computation close to data in general purpose Function-as-a-Service (FaaS) serverless computing. FaaS platforms enable a simple programming model with almost unbounded elasticity, making them an exciting choice for most cloud applications. However, current FaaS platforms achieve this...

show abstract

S/C: Speeding up Data Materialization with Bounded Memory

Park

2023

2023 IEEE 39th International Conference on Data Engineering (ICDE)

View full text Add to dashboard Cite

Mithril

Cited by 13 publications

References 15 publications

A data grouping model based on cache transaction for unstructured data storage systems

A data grouping model based on cache transaction for unstructured data storage systems

Informed optimization of cloud applications

S/C: Speeding up Data Materialization with Bounded Memory

Contact Info

Product

Resources

About