Random Slicing

Abstract

Use policyThe full-text may be used and/or reproduced, and given to third parties in any format or medium, without prior permission or charge, for personal research or study, educational, or not-for-prot purposes provided that:• a full bibliographic reference is made to the original source • a link is made to the metadata record in DRO • the full-text is not changed in any way The full-text must not be sold in any format or medium without the formal permission of the copyright holders.Please consult the full DRO policy for further details. The ever-growing amount of data requires highly scalable storage solutions. The most flexible approach is to use storage pools that can be expanded and scaled down by adding or removing storage devices. To make this approach usable, it is necessary to provide a solution to locate data items in such a dynamic environment. This paper presents and evaluates the Random Slicing strategy, which incorporates lessons learned from table-based, rule-based, and pseudo-randomized hashing strategies and is able to provide a simple and efficient strategy that scales up to handle exascale data. Random Slicing keeps a small table with information about previous storage system insert and remove operations, drastically reducing the required amount of randomness while delivering a perfect load distribution.

On the Efficient Delivery and Storage of IoT Data in Edge–Fog–Cloud Environments

“…A static distribution scheme distribution such as RUSH [36] is a family of algorithms that solves the scalability problem by facilitating the distribution of multiple replica objects among thousands of object-based storage devices. Random Slice (RS) [37] is a data distribution strategy that incorporates lessons learned from table-based and pseudo-random hashing strategies to be fair and efficient in homogeneous and heterogeneous environments to adapt and change storage containers. CRUSH [38] is a pseudo-random data distribution algorithm that efficiently and robustly distributes replicas across heterogeneous and structured clusters.…”

Erasure-Coding-Based Storage and Recovery for Distributed Exascale Storage Systems

“…Typically, studies on optimization of EC setting or minimization of network contention have been conducted when applying EC as a parallel processing basis. Studies on the optimization of EC setups refer to a setup of stripes appropriate to data I/O size during parallel I/O or efficient running of degraded I/O by diversifying the range of data encoding [49,50]. Studies on the minimization of network contention refer to minimization of network bottleneck by dividing a recovery operation into multiple parallel small suboperations [51].…”