Scheduling data-intensive bags of tasks in P2P grids with bittorrent-enabled data distribution

Briquet, Cyril; Dalem, Xavier; Jodogne, Sébastien; Marneffe, Pierre-Arnoul de

doi:10.1145/1272980.1272990

Cited by 9 publications

(4 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, the proposed Cloud scheduling heuristics are focused on compute-intensive BoTs [8], i.e., BoTs that mostly require computing resources (e.g., powerful processors) to be executed, and thus overhead times associated to data transfer and data storage are disregarded. However, when dealing with dataintensive BoTs [8], i.e., BoTs handling and transferring considerably large amounts of data, the overhead times associated to data transfers should be taken into account, such as in [34] to efficiently schedule and execute data-intensive BoTs.…”

Section: Discussionmentioning

confidence: 99%

A family of heuristics for agent-based elastic Cloud bag-of-tasks concurrent scheduling

Gutiérrez-García

Sim

2013

Future Generation Computer Systems

View full text Add to dashboard Cite

show abstract

Section: Discussionmentioning

confidence: 99%

A family of heuristics for agent-based elastic Cloud bag-of-tasks concurrent scheduling

Gutiérrez-García

Sim

2013

Future Generation Computer Systems

View full text Add to dashboard Cite

show abstract

“…In BitTorrent data are truncated into small pieces and a user performing data downloading as well as contributing simultaneously. This method can be incorporated into P2P-DCG for highly efficient task distribution [15,44,76]. However it also introduces security threats [32], which can be leveraged by using coding schemes with partial knowledge, which will be discussed later in Sect.…”

Section: Task Managementmentioning

confidence: 99%

A taxonomy of peer-to-peer desktop grid paradigms

Han

Liu

2010

Cluster Comput

View full text Add to dashboard Cite

Desktop grid systems and applications have generated significant impacts on science and engineering. The emerging convergence of grid and peer-to-peer (P2P) computing technologies further opens new opportunities for enabling P2P Desktop Grid systems. This paper presents a taxonomy for classifying P2P desktop grid implementation paradigms, aiming to summarize the state-of-the-art technologies and explore the current and potential solution space. To have a comprehensive taxonomy for P2P desktop grid paradigms, we investigate both computational and data grid systems. Moreover, to ease the understanding, the taxonomy is applied to selected case studies of P2P desktop grid systems. The taxonomy is expected to be used as a survey of the stateof-the-art, a design map, a guideline for novice researchers, a common vocabulary, or a design space for simulation and benchmark, and to be extended as the technologies rapidly evolve.

show abstract

“…At any time, at most one task can be run by a given resource. A highly scalable data transfer architecture [8] ensures the efficient transfer of input data files across the P2P Grid.…”

Section: Scheduling Modelmentioning

confidence: 99%

“…Data transfers are currently not taken into account. Simulating data transfers will considerably improve simulation accuracy when dealing with Data-Intensive BoTs [8].…”

Section: Future Workmentioning

confidence: 99%

Reproducible testing of distributed software with middleware virtualization and simulation

Briquet

Marneffe

2008

Proceedings of the 6th Workshop on Parallel and Distributed Systems: Testing, Analysis, and Debugging

Self Cite

View full text Add to dashboard Cite

P2P Grids are Grids organized into P2P networks where participant exchange computing time so as to complete computational tasks. Evaluating the performance of scheduling algorithms enables one to deploy those that are efficient. Performance is often evaluated experimentally or through simulation because these algorithms (typically heuristics) are too complex to model analytically. Testing the implementation of P2P Grid middleware before it is deployed is also important: Reproducing configurations or conditions that lead to unexpected outcomes is thus valuable.A P2P Grid environment exhibits multiple sources of failure and is typically dynamic and uncontrollable. Reproducing even basic behavior of Grid nodes in a controllable and repeatable manner is thus exceedingly difficult. Such lack of control over the environment is a major challenge in the software engineering of P2P Grid middleware [7]. Simulators have been proposed to evaluate the performance of scheduling algorithms, but are often limited in scope, reusability and accuracy, i.e. they rely on simplified models.We introduce a software engineering pattern -that we call code once, deploy twice -to both reduce the distance between simulated and implemented algorithms and reproduce, at will, Grid configurations and environments: A simulator implementation of a Grid architecture is built by virtualizing its middleware implementation. An immediate benefit is that most of the code can be reused between both implementations; only communications between Grid nodes, multithreading within Grid nodes and actual task execution are coded differently. As a derived benefit, most of the code of the middleware can be tested within the controlled environment of the simulator, before it is deployed as-is. Another benefit is high simulation accuracy. We describe the implementation of a P2P Grid following the code once, deploy twice pattern, that we believe is also relevant to other Grid types (certainly Volunteer Grids [5,4] and Desktop Grids [22], and possibly Globus-based Grids [3]).

show abstract

Scheduling data-intensive bags of tasks in P2P grids with bittorrent-enabled data distribution

Cited by 9 publications

References 18 publications

A family of heuristics for agent-based elastic Cloud bag-of-tasks concurrent scheduling

A family of heuristics for agent-based elastic Cloud bag-of-tasks concurrent scheduling

A taxonomy of peer-to-peer desktop grid paradigms

Reproducible testing of distributed software with middleware virtualization and simulation

Contact Info

Product

Resources

About