Phoenix: Memory Speed HPC I/O with NVM

Fernando, Pradeep; Kannan, Sudarsun; Gavrilovska, Ada; Schwan, Karsten

doi:10.1109/hipc.2016.023

Cited by 11 publications

(6 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We will explore pre-copying pages, based on write set estimation algorithms, into VAS page tables before the process restore. Our prior research has already demonstrated the practicality of such solutions for different workload classes [6,10]. In addition, information about hot pages could be stored in the snapshot, and could be used for page pre-copy.…”

Section: Summary and Future Workmentioning

confidence: 99%

Fast in-memory CRIU for docker containers

Venkatesh

Smejkal

Gavrilovska

2019

Proceedings of the International Symposium on Memory Systems

Self Cite

View full text Add to dashboard Cite

Server systems with large amounts of physical memory can benefit from using some of the available memory capacity for in-memory snapshots of the ongoing computations. In-memory snapshots are useful for services such as scaling of new workload instances, debugging, during scheduling, etc., which do not require snapshot persistence across node crashes/reboots. Since increasingly more frequently servers run containerized workloads, using technologies such as Docker, the snapshot, and the subsequent snapshot restore mechanisms, would be applied at granularity of containers. However, CRIU, the current approach to snapshot/restore containers, suffers from expensive filesystem write/read operations on image files containing memory pages, which dominate the runtime costs and impact the potential benefits of manipulating in-memory process state. In this paper, we demonstrate that these overheads can be eliminated by using MVAS-kernel support for multiple independent virtual address spaces (VAS), designed specifically for machines with large memory capacities. The resulting VAS-CRIU stores application memory as a separate snapshot address space in DRAM and avoids costly file system operations. This accelerates the snapshot/restore of address spaces by two orders of magnitude, resulting in an overall reduction in snapshot time by up to 10× and restore time by up to 9×. We demonstrate the utility of VAS-CRIU for container management services such as fine-grained snapshot generation and container instance scaling.

show abstract

Section: Summary and Future Workmentioning

confidence: 99%

Fast in-memory CRIU for docker containers

Venkatesh

Smejkal

Gavrilovska

2019

Proceedings of the International Symposium on Memory Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…At the same time their utilization is likely to suffer if they are placed on a compute blade, while rack scale disaggregation could provide better utilization. Another issue relates to the fact that most DCs would like to keep a copy of the data outside of the rack for resiliency purposes, but this can add significant overhead and remains a subject of research [103].…”

Section: Prototypementioning

confidence: 99%

Software-Defined “Hardware” Infrastructures: A Survey on Enabling Technologies and Open Research Directions

Roozbeh

Soares

Maguire

et al. 2018

IEEE Commun. Surv. Tutorials

View full text Add to dashboard Cite

show abstract

“…We reconcile this by taking advantage of the combined aggregate, local and remote memory capacity and accompanying bandwidth. The resulting technique, described in detail in [13], splits the output, partially writes portion of it out to NVRAM, and temporarily writes out a portion to DRAM staging buffers. The fault-tolerance of the DRAM-resident data is achieved through other techniques, specifically replication.…”

Section: Design Componentsmentioning

confidence: 99%

“…The runtime remains responsible for dynamically assisting the overall resource availability and application demands in order to parametrize the execution of the technique, including the split ratio, or the details of realizing the persistence of the staging buffers, and it does this given target optimization metrics, such as gains in execution time, energy efficiency, or reliability guarantees. In addition, we integrate and further extend additional optimizations to reduce the overall data movement requirements, those taking place in the critical path of persistence operations, or both [13,21], and will further consider methods for integrating energy awareness leveraging our earlier work [22].…”

Section: Design Componentsmentioning

confidence: 99%

Unity

Jones

Brim

Vallée

et al. 2017

Proceedings of the 7th International Workshop on Runtime and Operating Systems for Supercomputers ROSS 2017

Self Cite

View full text Add to dashboard Cite

This paper describes the vision for UNITY , a new high-performance computing focused data storage abstraction that places the entire memory hierarchy, including both traditionally separated memoryand file-based data storage, into one storage continuum. Through the use of a novel API and a set of services centered around a smart runtime system, UNITY is able to provide a number of valuable and interesting benefits. The unified storage space provides a scalable and resilient data environment that dynamically manages the mapping of data onto available resources based on multiple factors, including desired persistence and energy budget considerations. By eliminating the need for high-performance computing domain scientists to develop architecture-dependent optimizations for rapidly evolving data storage technologies, UNITY addresses both ease-of-use and performance.

show abstract

Phoenix: Memory Speed HPC I/O with NVM

Cited by 11 publications

References 15 publications

Fast in-memory CRIU for docker containers

Fast in-memory CRIU for docker containers

Software-Defined “Hardware” Infrastructures: A Survey on Enabling Technologies and Open Research Directions

Unity

Contact Info

Product

Resources

About