Toward a theory for scheduling dags in Internet-based computing

Malewicz, Grzegorz; Rosenberg, Arnold L.; Yurkewych, M.

doi:10.1109/tc.2006.91

Cited by 51 publications

(131 citation statements)

References 10 publications

Supporting

Mentioning

131

Contrasting

Order By: Relevance

“…Our first strategy (Section 3.2.1.A) creates a clustered task of any desired size, by exploiting just the order of task-executions of (any) one of the computation's IC-optimal schedules. All of our other strategies exploit aspects of the detailed structure of the computation, as exposed by the suite of algorithms developed in [15] for crafting IC-optimal schedules. These strategies range from ones that work on any computation that admits an IC-optimal schedule (Sections 3.2.1B, 3.2.2) to ones that depend on the very detailed structure of the computation (Sections 3.3.2, 3.3.3).…”

Section: Resultsmentioning

confidence: 99%

“…Note first that we lose no generality by seeking an IC-optimal schedule for a among schedules that execute all of 's nonsinks before any of its sinks [15].…”

Section: Ic-optimal Schedules Via D Structurementioning

confidence: 99%

“…[13])and aggressively multi-core and proposed exascale architectures. Ongoing work [4,6,[15][16][17] has been developing IC-scheduling, a master-worker framework for executing static computations that have intertask dependencies (modeled as s) in a way that enhances the rate at which a computation-'s tasks are rendered eligible for execution. IC-scheduling is motivated by the intuition that rendering tasks eligible as fast as possiblea purely -theoretic goalwill enhance the performance of a "task-hungry" platform, no matter what the (instantaneous) distribution of its constituent computer's speeds.…”

Section: Introductionmentioning

confidence: 99%

“…Both studies suggest that IC-scheduling can accelerate a broad range of computations under a broad range of worker-arrival patterns, by 5% − 30%, or more. IC-scheduling has evolved from the case studies of [16,17] to the algorithmic framework of [4,6,15], and it can now schedule IC-optimallyi.e., in a manner that produces eligible tasks optimally quicklya large repertoire of significant "real" computations; cf. [5].…”

Section: Introductionmentioning

confidence: 99%

“…We strive for techniques that allow us to choose the size of a clustered taskhence, its coarsenessdynamically, to acknowledge that we often do not know a priori what resources a worker will have at each of its arrivals. Restricting attention to computations that admit IC-optimal schedules limits significantly the class of computations that our strategies can deal with (many computations do not admit any IC-optimal schedule [15])but this class has been shown to include a broad range of (especially) scientific real computations, including, e.g., all of those discussed in [5]. In compensation, we provide nonobvious, computationally beneficial clustering strategies for computations that do admit IC-optimal schedules.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

On clustering DAGs for task-hungry computing platforms

Cordasco

Rosenberg

Sims

2011

Open Computer Science

View full text Add to dashboard Cite

Many modern computing platforms are "task-hungry": their performance is enhanced by always having as many tasks available for execution as possible. IC-scheduling, a master-worker framework for executing static computations that have intertask dependencies (modeled as s), was developed with precisely the goal of rendering a computation-'s tasks eligible for execution at the maximum possible rate. The current paper addresses the problem of enhancing IC-scheduling so that it can accommodate the varying computational resources of different workers, by clustering a computation-'s tasks, while still producing eligible (now, clustered) tasks at the maximum possible rate. The task-clustering strategies presented exploit the structure of the computation being performed, ranging from a strategy that works for any , to ones that build increasingly on the explicit structure of the being scheduled. Keywords

show abstract

Section: Resultsmentioning

confidence: 99%

“…Note first that we lose no generality by seeking an IC-optimal schedule for a among schedules that execute all of 's nonsinks before any of its sinks [15].…”

Section: Ic-optimal Schedules Via D Structurementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

On clustering DAGs for task-hungry computing platforms

Cordasco

Rosenberg

Sims

2011

Open Computer Science

View full text Add to dashboard Cite

show abstract

On constructing DAG‐schedules with large areas

Roche

Rosenberg

Rajaraman

2015

Concurrency and Computation

View full text Add to dashboard Cite

Abstract. The Area of a schedule Σ for a DAG G is a quality metric that measures the rate at which Σ renders G's nodes eligible for execution. Specifically, AREA(Σ) is the average number of nodes of G that are eligible for execution as Σ executes G node by node. Extensive simulations suggest that, for many distributions of processor availability and power, DAG-schedules having larger Areas execute DAGs faster on platforms that are dynamically heterogeneous: the platform's processors change power and availability status in unpredictable ways and at unpredictable times. (Clouds and desktop grids exemplify such platforms.) While Area-maximal schedules can provably be found for every DAG, efficient generators of such schedules are known only for families of well-structured DAGs. Our first result shows that the problem of crafting Area-maximal schedules for general DAGs is NP-complete, hence likely computationally intractable. The lack of efficient Area-maximizing schedulers for general DAGs has instigated the development of several heuristics for producing DAG-schedules that have large Areas. We propose a novel polynomial-time heuristic that produces schedules having quite large Areas; the heuristic is based on the Sidney decomposition of a DAG. (1) Simulations on DAGs having random structure yield the following results. The Sidney heuristic produces schedules whose Areas: (a) are at least 85% of maximal; (b) are at least 1.25 times greater than previously known heuristics. (2) Simulations on DAGs having the structure of random "LEGO R " DAGs (as formulated in earlier studies) indicate that the schedules produced by the Sidney heuristic have Areas that are at least 1.5 times greater than previously known heuristics. The "85%" result is obtained from formulating the Areamaximization problem as a Linear Program (LP); the Areas of DAG-schedules produced by the Sidney heuristic are at least 85% of the Area-value produced by the (unrounded) LP. (3) The reported results on random DAGs are essentially matched by a second heuristic, which produces DAG-schedules by rounding the results of the LP formulation.

show abstract

Changing Challenges for Collaborative Algorithmics

Rosenberg

Handbook of Nature-Inspired and Innovative Computing

View full text Add to dashboard Cite

Toward a theory for scheduling dags in Internet-based computing

Cited by 51 publications

References 10 publications

On clustering DAGs for task-hungry computing platforms

On clustering DAGs for task-hungry computing platforms

On constructing DAG‐schedules with large areas

Changing Challenges for Collaborative Algorithmics

Contact Info

Product

Resources

About