Predicting file lifetimes for data placement in multi-tiered storage systems for HPC

Thomas, Luis; Gougeaud, Sebastien; Rubini, Stéphane; Deniel, Philippe; Boukhobza, Jalil

doi:10.1145/3439839.3458733

Cited by 8 publications

(10 citation statements)

References 23 publications

(2 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In previous work [34], the authors used the access path of a file to accurately predict when a file will last be accessed. They used a single Convolutional Neural Network with a custom loss function in order to reduce the computing power necessary for a prediction.…”

Section: File Lifetime Predictionmentioning

confidence: 99%

“…As described earlier, SLRL is based on a file lifetime prediction model that was described in a previous work. This model is able to predict the lifetime of a file based on its absolute path [34] with an 98% accuracy.…”

Section: Slrl Policymentioning

confidence: 99%

“…Indeed, it is as simple as FIFO in terms of metadata usage, while being comparable to LRU in terms of hit rate. SLRL uses file lifetimes predicted based on the study in [34]. The different components of file paths were used in order to predict file lifetimes at their creation.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

SLRL

Nicolas

Thomas

Hadjadj‐Aoul

et al. 2022

SIGOPS Oper. Syst. Rev.

View full text Add to dashboard Cite

HPC systems are composed of multiple tiers of storage, from the top high performance tier (high speed SSDs) to the bottom capacitive one (tapes). File placement in such architecture is managed through prefetchers (bottom-up) and eviction policies (top-down). Most state-of-the-art work focus on the former while using algorithm flavors of LRU, LFU and FIFO for the latter. LRU was for long considered the best choice. However, recent studies has shown that the simplicity of FIFO could make it more scalable than LRU because of metadata management, and thus more adequate in several cases. In this paper, we propose a new eviction policy based on predicted files lifetimes. It is comparable to FIFO in terms of metadata overhead and simplicity (thus scalability), while giving a hit ratio comparable to LRU (or even 10% better for some tested traces). We also propose a naive multi-tier heterogeneous storage simulator implementation to evaluate such policies.

show abstract

Section: File Lifetime Predictionmentioning

confidence: 99%

Section: Slrl Policymentioning

confidence: 99%

See 1 more Smart Citation

SLRL

Nicolas

Thomas

Hadjadj‐Aoul

et al. 2022

SIGOPS Oper. Syst. Rev.

View full text Add to dashboard Cite

show abstract

“…Storage The objective of the storage phase is to save data securely throughout the life cycle. Data storage is an essential process of big data analytics in real-world applications [65,166]. We noticed in the literature that the storage phase is considered in all data lifecycles models.…”

Section: Analysis Phase Key Functionsmentioning

confidence: 99%

“…• Store data in an appropriate location (in-house data center or private cloud environment) in a secure, scalable, accessible, and reliable manner [65,90,147]. • Compliance with industry standards, a) to store GBD along with improved data structures, appropriate cloud data security, and backing fault tolerance; and b) to improve data storage systems performance in terms of capacity and speed [65,75,142,166]. • Implement a plan for the selection of modernized and extensible tools for data storage along with a balanced approach for data availability and scalability [58,64,95].…”

Section: Storage Phase Key Functionsmentioning

confidence: 99%

DaLiF: a data lifecycle framework for data-driven governments

2021

View full text Add to dashboard Cite

The public sector, private firms, business community, and civil society are generating data that is high in volume, veracity, velocity and comes from a diversity of sources. This kind of data is known as big data. Public Administrations (PAs) pursue big data as “new oil” and implement data-centric policies to transform data into knowledge, to promote good governance, transparency, innovative digital services, and citizens’ engagement in public policy. From the above, the Government Big Data Ecosystem (GBDE) emerges. Managing big data throughout its lifecycle becomes a challenging task for governmental organizations. Despite the vast interest in this ecosystem, appropriate big data management is still a challenge. This study intends to fill the above-mentioned gap by proposing a data lifecycle framework for data-driven governments. Through a Systematic Literature Review, we identified and analysed 76 data lifecycles models to propose a data lifecycle framework for data-driven governments (DaliF). In this way, we contribute to the ongoing discussion around big data management, which attracts researchers’ and practitioners’ interest.

show abstract