Memory management for billions of small objects in a distributed in-memory storage

Klein, Florian; Beineke, Kevin; Schöttner, Michael

doi:10.1109/cluster.2014.6968771

Cited by 10 publications

(3 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All-replication stores multiple replicas for each object, including the key, value, metadata, and reference to the object. It has been used in Facebook's Memcache [40], DXRAM [35], RAMCloud [41], and various open-source in-memory KV stores (e.g., [2,4,8,10]). If there is a failed object, the KV store retrieves its replica from another working server.…”

Section: Existing Data Modelsmentioning

confidence: 99%

See 1 more Smart Citation

Erasure coding for small objects in in-memory KV storage

Yiu

Chan

Lee

2017

Proceedings of the 10th ACM International Systems and Storage Conference

View full text Add to dashboard Cite

We present MemEC, an erasure-coding-based inmemory key-value (KV) store that achieves high availability and fast recovery while keeping low data redundancy across storage servers. MemEC is specifically designed for workloads dominated by small objects. By encoding objects in entirety, MemEC is shown to incur 60% less storage redundancy for small objects than existing replication-and erasure-coding-based approaches. It also supports graceful transitions between decentralized requests in normal mode (i.e., no failures) and coordinated requests in degraded mode (i.e., with failures). We evaluate our MemEC prototype via testbed experiments under read-heavy and update-heavy YCSB workloads. We show that MemEC achieves high throughput and low latency in both normal and degraded modes, and supports fast transitions between the two modes.

show abstract

Section: Existing Data Modelsmentioning

confidence: 99%

“…Failures are prevalent in distributed storage [28]. To maintain data availability, most existing in-memory KV stores replicate data copies across storage servers [2,4,8,10,35,40]. However, replication incurs high storage overhead and multiplies the memory cost, which is still much higher than the cost of traditional disk-based storage.…”

Section: Introductionmentioning

confidence: 99%

Erasure coding for small objects in in-memory KV storage

Yiu

Chan

Lee

2017

Proceedings of the 10th ACM International Systems and Storage Conference

View full text Add to dashboard Cite

show abstract

“…Distributed memory, where the memory is close to the point at which it is used offers huge benefits, so long as the memories are kept coherent where necessary [19,21,13].…”

Section: Introductionmentioning

confidence: 99%

A distributed memory, local configuration technique for re-configurable logic designs

Beasley¹

2020

Preprint

View full text Add to dashboard Cite

The use and location of memory in integrated circuits plays a key factor in their performance. Memory requires large physical area, access times limit overall system performance and connectivity can result in large fan-out. Modern FPGA systems and ASICs contain an area of memory used to set the operation of the device from a series of commands set by a host. Implementing these settings registers requires a level of care otherwise the resulting implementation can result in a number of large fan-out nets that consume valuable resources complicating the placement of timing critical pathways. This paper presents an architecture for implementing and programming these settings registers in a distributed method across an FPGA and how the presented architecture works in both clock-domain crossing and dynamic partial re-configuration applications. The design is compared to that of a 'global' settings register architecture. We implement the architectures using Intel FPGAs Quartus Prime software targeting an Intel FPGA Cyclone V. It is shown that the distributed memory architecture has a smaller resource cost (as small as 25% of the ALMs and 20% of the registers) compared to the global memory architectures.

show abstract