Mean value technique for closed fork-join networks

Varki, Elizabeth

doi:10.1145/301453.301484

Cited by 46 publications

(15 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Given the difficulties posed by single-stage fork-join systems, few works consider multi-stage networks. A notable exception is [10] where an approximation for closed fork-join networks is developed.…”

Section: Introductionmentioning

confidence: 99%

Non-asymptotic delay bounds for (k, l) fork-join systems and multi-stage fork-join networks

Fidler

Jiang²

2016

IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications

View full text Add to dashboard Cite

Parallel systems have received increasing attention with numerous recent applications such as fork-join systems, load-balancing, and l-out-of-k redundancy. Common to these systems is a join or resequencing stage, where tasks that have finished service may have to wait for the completion of other tasks so that they leave the system in a predefined order. These synchronization constraints make the analysis of parallel systems challenging and few explicit results are known. In this work, we model parallel systems using a max-plus approach that enables us to derive statistical bounds of waiting and sojourn times. Taking advantage of max-plus system theory, we also show end-to-end delay bounds for multi-stage fork-join networks. We contribute solutions for basic G|G|1 fork-join systems, parallel systems with load-balancing, as well as general (k, l) fork-join systems with redundancy. Our results provide insights into the respective advantages of l-out-of-k redundancy vs. load-balancing.

show abstract

Section: Introductionmentioning

confidence: 99%

Non-asymptotic delay bounds for (k, l) fork-join systems and multi-stage fork-join networks

Fidler

Jiang²

2016

IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications

View full text Add to dashboard Cite

show abstract

“…Fork/join-based [1]: We consider the execution of a parallel-phase as a fork-join block, and use previously adopted estimates of the average response time of fork/joins. One such estimate is the product of the k − th harmonic function by the maximum average response time of k tasks [13].…”

Section: Estimate Task Response Timementioning

confidence: 99%

Mapreduce performance model for Hadoop 2.x

Glushkova

Jovanovic

Abelló

2019

Information Systems

View full text Add to dashboard Cite

MapReduce is a popular programming model for distributed processing of large data sets. Apache Hadoop is one of the most common open-source implementations of such paradigm. Performance analysis of concurrent job executions has been recognized as a challenging problem, at the same time, that may provide reasonably accurate job response time estimation at significantly lower cost than experimental evaluation of real setups. In this paper, we tackle the challenge of defining MapReduce performance model for Hadoop 2.x. While there are several efficient approaches for modeling the performance of MapReduce workloads in Hadoop 1.x, they could not be applied to Hadoop 2.x due to fundamental architectural changes and dynamic resource allocation in Hadoop 2.x. Thus, the proposed solution is based on an existing performance model for Hadoop 1.x, but taking into consideration architectural changes and capturing the execution flow of a MapReduce job by using queuing network model. This way, the cost model reflects the intra-job synchronization constraints that occur due the contention at shared resources. The accuracy of our solution is validated via comparison of our model estimates against measurements in a real Hadoop 2.x setup.

show abstract

“…(2.2) Closed QNMs: Closed QNMs process a fixed number of jobs exhibiting internal concurrency in the form of F/J requests [Heidelberger and Trivedi 1983;Liu and Perros 1991;Varki and Dowdy 1996;Varki 1999;Varki 2001;Casale et al 2008;Varki et al 2013]. A phase-type delay is considered as part of a closed QNM in Krishnamurthy et al [2004].…”

Section: Overview: Fork/join and Related Queueing Systemsmentioning

confidence: 99%

Analysis of Fork/Join and Related Queueing Systems

Thomasian¹

2014

ACM Comput. Surv.

View full text Add to dashboard Cite

Fork/join (F/J) requests arise in contexts such as parallel computing, query processing in parallel databases, and parallel disk access in RAID. F/J requests spawn K tasks that are sent to K parallel servers, and the completion of all K tasks marks the completion of an F/J request. The exact formula for the mean response time of K = 2-way F/J requests derived under Markovian assumptions (R F/J 2 ) served as the starting point for an approximate expression for R F/J K for 2 < K ≤ 32. When servers process independent requests in addition to F/J requests, the mean response time of F/J requests is better approximated by R max K , which is the maximum of the response times of tasks constituting F/J requests. R max K is easier to compute and serves as an upper bound to R F/J K . We discuss techniques to compute R max K and generally the maximum of K random variables denoting the processing times of the tasks of a parallel computation X max K . Graph models of computations such as Petri nets-a more general form of parallelism than F/J requests-are also discussed in this work. Jobs with precedence constraints may require multiple resources, which are represented by a queueing network model. We also discuss various queueing systems related to F/J queueing systems and outline their analysis. ACM Reference Format:Alexander Thomasian. 2014. Analysis of fork-join and related queueing systems.

show abstract

Mean value technique for closed fork-join networks

Cited by 46 publications

References 15 publications

Non-asymptotic delay bounds for (k, l) fork-join systems and multi-stage fork-join networks

Non-asymptotic delay bounds for (k, l) fork-join systems and multi-stage fork-join networks

Mapreduce performance model for Hadoop 2.x

Analysis of Fork/Join and Related Queueing Systems

Contact Info

Product

Resources

About