Replication under scalable hashing:a family of algorithms for scalable decentralized data distribution

Honicky, R.J.; Miller, Ethan L.

doi:10.1109/ipdps.2004.1303042

Cited by 69 publications

(53 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The first methods with dedicated support for replication were proposed by Honicky and Miller [Honicky and Miller 2003] [Honicky and Miller 2004]. RUSH (Replication Under Scalable Hashing) maps replicated objects to a scalable collection of storage servers according to user-specified server weighting.…”

Section: Previous Resultsmentioning

confidence: 99%

“…The RUSH algorithms all proceed in two stages, first identifying the appropriate cluster in which to place an object, and then identifying the disk within a cluster [Honicky and Miller 2003] [Honicky and Miller 2004]. Within a cluster, replicas assigned to the cluster are mapped to disks using prime number arithmetic that guarantees that no two replicas of a single object can be mapped to the same disk.…”

Section: Rushmentioning

confidence: 99%

See 1 more Smart Citation

Random Slicing

et al. 2014

View full text Add to dashboard Cite

Use policyThe full-text may be used and/or reproduced, and given to third parties in any format or medium, without prior permission or charge, for personal research or study, educational, or not-for-prot purposes provided that:• a full bibliographic reference is made to the original source • a link is made to the metadata record in DRO • the full-text is not changed in any way The full-text must not be sold in any format or medium without the formal permission of the copyright holders.Please consult the full DRO policy for further details. The ever-growing amount of data requires highly scalable storage solutions. The most flexible approach is to use storage pools that can be expanded and scaled down by adding or removing storage devices. To make this approach usable, it is necessary to provide a solution to locate data items in such a dynamic environment. This paper presents and evaluates the Random Slicing strategy, which incorporates lessons learned from table-based, rule-based, and pseudo-randomized hashing strategies and is able to provide a simple and efficient strategy that scales up to handle exascale data. Random Slicing keeps a small table with information about previous storage system insert and remove operations, drastically reducing the required amount of randomness while delivering a perfect load distribution.

show abstract

Section: Previous Resultsmentioning

confidence: 99%

Section: Rushmentioning

confidence: 99%

Random Slicing

et al. 2014

View full text Add to dashboard Cite

show abstract

“…The first methods with dedicated support for replication were proposed by Honicky and Miller [14] [15]. RUSH (Replication Under Scalable Hashing) maps replicated objects to a scalable collection of storage servers according to userspecified server weighting.…”

Section: B Previous Resultsmentioning

confidence: 99%

Reliable and randomized data distribution strategies for large scale storage systems

Miranda

Effert

Kang

et al. 2011

2011 18th International Conference on High Performance Computing

View full text Add to dashboard Cite

Abstract-The ever-growing amount of data requires highly scalable storage solutions. The most flexible approach is to use storage pools that can be expanded and scaled down by adding or removing storage devices. To make this approach usable, it is necessary to provide a solution to locate data items in such a dynamic environment. This paper presents and evaluates the Random Slicing strategy, which incorporates lessons learned from table-based, rule-based, and pseudo-randomized hashing strategies and is able to provide a simple and efficient strategy that scales up to handle exascale data. Random Slicing keeps a small table with information about previous storage system insert and remove operations, drastically reducing the required amount of randomness while delivering a perfect load distribution.

show abstract

“…Although in our experiments we allocate data from scratch when new disks arrive, it should be pointed out that a number of algorithms have been proposed and implemented which are able to perform a reorganization with minimum overhead (please see, e.g., [19,20,21,5]). …”

Section: Dynamically Increasing the Number Of Binsmentioning

confidence: 99%

Balls into non-uniform bins

Berenbrink

Brinkmann

Friedetzky

et al. 2014

Journal of Parallel and Distributed Computing

View full text Add to dashboard Cite

Citation for published item:feren rinkD etr nd frinkm nnD endr¡ e nd priedetzkyD om nd x gelD v rs @PHIRA 9f lls into nonEuniform insF9D tourn l of p r llel nd distri uted omputingFD UR @PAF ppF PHTSEPHUTFFurther information on publisher's website: Use policyThe full-text may be used and/or reproduced, and given to third parties in any format or medium, without prior permission or charge, for personal research or study, educational, or not-for-pro t purposes provided that:• a full bibliographic reference is made to the original source • a link is made to the metadata record in DRO• the full-text is not changed in any way The full-text must not be sold in any format or medium without the formal permission of the copyright holders.Please consult the full DRO policy for further details. Abstract Balls-into-bins games for uniform bins are widely used to model randomised load balancing strategies. Recently, balls-into-bins games have been analysed under the assumption that the selection probabilities for bins are not uniformly distributed. These new models are motivated by properties of many peer-topeer (P2P) networks. In this paper we consider scenarios in which non-uniform selection probabilities help to balance the load among the bins. While previous evaluations try to find strategies for identical bins, we investigate heterogeneous bins where the "capacities" of the bins might differ significantly. We look at the allocation of m balls into n bins of total capacity C where each ball has d random bin choices. For such heterogeneous environments we show that the maximum load remains bounded by ln ln(n)/ ln(d) + O(1) w.h.p. if the number of balls m equals the total capacity C. Further analytical and simulative results show better bounds and values for the maximum loads in special cases.

show abstract

Replication under scalable hashing:a family of algorithms for scalable decentralized data distribution

Abstract: Abstract

Cited by 69 publications

References 19 publications

Random Slicing

Random Slicing

Reliable and randomized data distribution strategies for large scale storage systems

Balls into non-uniform bins

Contact Info

Product

Resources

About