Load Balancing Prioritized Tasks via Work-Stealing

Imam, Shams; Sarkar, Vivek

doi:10.1007/978-3-662-48096-0_18

Cited by 8 publications

(9 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…asynchronous one-sided WS with distributed decen-work-commu-commu-PGAS MPI global load RTM memory tralized stealing nication nication RMA information Barros et al [24] Andreolli et al [26] Andreolli et al [27] Sena et al [28] x Hofmeyr et al [29] x Tchiboukdjian et al [30] x Imam and Sarkar [31] x x x Khaitan et al [32] x Tesser et al [33] x Tesser et al [34] x Tesser et al [35] x Padoin et al [36] x Padoin et al [37] x Sharma and Kanungo [38] x x x Zheng et al [39] x x x Martinez et al [40] x x x x Khaitan and Mccalley [41] x x x x Mor and Maillard [42] x x x x Li et al [43] x x x x x x Kumar et al [44] x x x x x x Dinan et al [21] x x x x x x Vishnu and Agarwal [49] x…”

Section: Discussionmentioning

confidence: 99%

Distributed-Memory Load Balancing With Cyclic Token-Based Work-Stealing Applied to Reverse Time Migration

et al. 2019

View full text Add to dashboard Cite

Reverse time migration (RTM) is a prominent technique in seismic imaging. Its resulting subsurface images are used in the industry to investigate with higher confidence the existence and the conditions of oil and gas reservoirs. Because of its high computational cost, RTM must make use of parallel computers. Balancing the workload distribution of an RTM is a growing challenge in distributed computing systems. The competition for shared resources and the differently-sized tasks of the RTM are some of the possible sources of load imbalance. Although many load balancing techniques exist, scaling up for large problems and large systems remains a challenge because synchronization overhead also scales. This paper proposes a cyclic token-based work-stealing (CTWS) algorithm for distributed memory systems applied to RTM. The novel cyclic token approach reduces the number of failed steals, avoids communication overhead, and simplifies the victim selection and the termination strategy. The proposed method is implemented as a C library using the one-sided communication feature of the message passing interface (MPI) standard. Results obtained by applying the proposed technique to balance the workload of a 3D RTM system present a factor of 14.1 % speedup and reductions of the load imbalance of 78.4 % when compared to the conventional static distribution. 1The migration of seismic data is the process that attempts to build an image of the Earth's interior from recorded field data. Migration places these data into their actual geological position in the subsurface using numerical approximations of either wave-theoretical or ray-theoretical approaches to simulate the propagation of seismic waves [1].The wave-theoretical approach to the propagation of seismic waves employs the finite difference method (FDM) [2,3] to numerically solve the equation describing the movement of the waves [1,4]. This approach is prevalent among the geophysical community, due to its capacity of dealing with substantial velocity variations in complex geology (e.g., pre-salt).Reverse time migration (RTM) [5-9] implements this approach. It is one of the most known FDM-based migration methods. RTM is computationally intensive in terms of data storage and handling, and its use of high-complexity algorithms. Therefore, exploiting parallelism is mandatory for RTM implementations in 3D Earth models (3D RTM) [10].Parallel architectures can be classified as shared memory, when there is a single memory address space available to all processing units (e.g., nodes or cores), or distributed memory otherwise [11]. Many scientific and industrial computational resources are distributed memory systems composed of multiprocessor nodes with shared memory systems. A hybrid parallel application works at these two levels of parallelism. It can distribute the total workload among the nodes of a distributed memory system. Each node, then, distributes its subset of the workload among the processing units of its shared memory system. Parallel machines can also be described as het...

show abstract

Section: Discussionmentioning

confidence: 99%

Distributed-Memory Load Balancing With Cyclic Token-Based Work-Stealing Applied to Reverse Time Migration

et al. 2019

View full text Add to dashboard Cite

show abstract

“…All these schedulers can be viewed as cooperative schedulers because they don't treat threads as competing for the same resources but schedule them cooperatively with the goal of maximizing throughput. Other work has shown that scheduling based on priorities [Imam and Sarkar 2015;Wimmer et al 2013Wimmer et al , 2014 can improve throughput in some applications.…”

Section: Related Workmentioning

confidence: 99%

Fairness in responsive parallelism

Muller

Westrick

Acar

2019

Proc. ACM Program. Lang.

View full text Add to dashboard Cite

Research on parallel computing has historically revolved around compute-intensive applications drawn from traditional areas such as high-performance computing. With the growing availability and usage of multicore chips, applications of parallel computing now include interactive parallel applications that mix compute-intensive tasks with interaction, e.g., with the user or more generally with the external world. Recent theoretical work on responsive parallelism presents abstract cost models and type systems for ensuring and reasoning about responsiveness and throughput of such interactive parallel programs.In this paper, we extend prior work by considering a crucial metric: fairness. To express rich interactive parallel programs, we allow programmers to assign priorities to threads and instruct the scheduler to obey a notion of fairness. We then propose the fairly prompt scheduling principle for executing such programs; the principle specifies the schedule for multithreaded programs on multiple processors. For such schedules, we prove theoretical bounds on the execution and response times of jobs of various priorities. In particular, we bound the amount, i.e., stretch, by which a low-priority job can be delayed by higher-priority work. We also present an algorithm designed to approximate the fairly prompt scheduling principle on multicore computers, implement the algorithm by extending the Standard ML language, and present an empirical evaluation.

show abstract

“…We do wish to emphasize that standard work-stealing schedulers will not provide this type of work bounds, since they do not provide any guarantees in terms of the rank of elements removed: the rank becomes unbounded over long executions, since a single random queue is sampled at every stealing step [2]. To our knowledge, there is only one previous attempt to add priorities to work-stealing schedulers [17], using a multi-level global queue of tasks, partitioned by priority. This technique is different, and provides no work guarantees.…”

Section: Introductionmentioning

confidence: 99%

Relaxed Schedulers Can Efficiently Parallelize Iterative Algorithms

Alistarh

Brown

Kopinsky

et al. 2018

Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing

View full text Add to dashboard Cite

There has been significant progress in understanding the parallelism inherent to iterative sequential algorithms: for many classic algorithms, the depth of the dependence structure is now well understood, and scheduling techniques have been developed to exploit this shallow dependence structure for efficient parallel implementations. A related, applied research strand has studied methods by which certain iterative task-based algorithms can be efficiently parallelized via relaxed concurrent priority schedulers. These allow for high concurrency when inserting and removing tasks, at the cost of executing superfluous work due to the relaxed semantics of the scheduler.In this work, we take a step towards unifying these two research directions, by showing that there exists a family of relaxed priority schedulers that can efficiently and deterministically execute classic iterative algorithms such as greedy maximal independent set (MIS) and matching. Our primary result shows that, given a randomized scheduler with an expected relaxation factor of k in terms of the maximum allowed priority inversions on a task, and any graph on n vertices, the scheduler is able to execute greedy MIS with only an additive factor of poly(k) expected additional iterations compared to an exact (but not scalable) scheduler. This counter-intuitive result demonstrates that the overhead of relaxation when computing MIS is not dependent on the input size or structure of the input graph. Experimental results show that this overhead can be clearly offset by the gain in performance due to the highly scalable scheduler. In sum, we present an efficient method to deterministically parallelize iterative sequential algorithms, with provable runtime guarantees in terms of the number of executed tasks to completion. arXiv:1808.04155v1 [cs.DS] 13 Aug 2018 may only depend on a small subset of other nodes, namely its neighbors which are higher priority in the random order. Blelloch, Fineman and Shun [7] performed an in-depth study of the asymptotic properties of this dependence structure, proving that, for any graph, the maximal depth of a chain of dependences is in fact O(log 2 n) with high probability, where n is the number of nodes in the graph. Recently, an impressive analytic result by Fischer and Noever [13] provided tight Θ(log n) bounds on the maximal dependency depth for greedy sequential MIS, effectively closing this problem for MIS. Beyond greedy MIS, there has been significant progress in analyzing the dependency structure of other fundamental sequential algorithms, such as algorithms for matching [7], list contraction [25], Knuth shuffle [25], linear programming [5], and graph connectivity [5].An alternative approach has been to employ relaxed data structures to schedule task-based programs. Starting with Karp and Zhang [19], the general idea is that, in some applications, the scheduler can relax the strict order induced by following the sequential algorithm, and allow tasks to be processed speculatively ahead of their dependencies, without loss o...

show abstract

Load Balancing Prioritized Tasks via Work-Stealing

Cited by 8 publications

References 10 publications

Distributed-Memory Load Balancing With Cyclic Token-Based Work-Stealing Applied to Reverse Time Migration

Distributed-Memory Load Balancing With Cyclic Token-Based Work-Stealing Applied to Reverse Time Migration

Fairness in responsive parallelism

Relaxed Schedulers Can Efficiently Parallelize Iterative Algorithms

Contact Info

Product

Resources

About