Heterogeneous Distributed Computing

Abstract

One of the biggest challenges with high-performance computing is that as machine architectures become more advanced to obtain increased peak performance, only a small fraction of this performance is achieved on many real application sets. This is because a typical application may have various subtasks with different architectural requirements. When such an application is executed on a given machine, the machine spends most of its time executing subtasks for which they are unsuited. With the recent advances in high-speed digital communications, it has become possible to use collections of different high-performance machines in concert to solve computationally intensive application tasks. This article describes the issues involved with using such a heterogeneous computing (HC) suite of machines to solve application tasks.A hypothetical example application that has various subtasks, which are best suited, for different machine architectures is shown in Figure 1 (based on (FrS93)). The example application executes for 100 time units on a baseline serial machine. The application consists of four subtasks: the first is best suited to execute on an SIMD (synchronous) parallel machine, the second is best suited for a distributed-memory MIMD (asynchronous) parallel machine, the third is best suited for a shared-memory MIMD machine, and the fourth is best suited to execute on a vector (pipelined) machine.Executing the whole application on an SIMD machine may improve the execution time of the SIMD subtask from 25 to 0.01 time units, and the other subtasks to varying extents. The overall execution time improvement may only be about a factor of five because other subtasks may not be well suited for an SIMD machine. Using four different machines that match the computational requirements for each of the individual subtasks can result in an overall execution time that is

Dynamically mapping tasks with priorities and multiple deadlines in a heterogeneous environment

“…The estimated time to complete (ETC) values of each task on each machine is assumed to be known based on user-supplied information, experiential data, task profiling and analytical benchmarking, or other techniques (e.g., [1,15,16,21,26,40]). Determination of ETC values is a separate research problem, and the assumption of such ETC information is a common practice in mapping research (e.g., [16,20,21,24,31,39]).…”

Characterizing Task-Machine Affinity in Heterogeneous Computing Environments

“…Entry (i, j) in the ETC matrix represents the ETC of task type i on machine j. The ETC values can be based on user supplied information, experimental data, or task profiling and analytical benchmarking (e.g., [1,12,13,17,19,25,26]). The determination of ETC values is a separate research problem; the assumption of such ETC information is a common practice in resource allocation research (e.g., [5,7,10,13,15,17,18,22,24]).…”

Dynamic resource allocation heuristics that manage tradeoff between makespan and robustness

“…The estimated time to compute (ETC) values of each task on each machine are assumed to be known based on user supplied information, experiential data, task profiling and analytical benchmarking, or other techniques (e.g., [1,15,16,19,25,34]). Determination of ETC values is a separate research problem, and the assumption of such ETC information is a common practice in mapping research (e.g., [16,18,19,21,30,33]).…”