Multidimensional Static Block Data Decomposition for Heterogeneous Clusters

Kalinov, Alexey; Klimov, S.

doi:10.1007/978-3-540-24669-5_117

Cited by 6 publications

(5 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Kalinov and Klimov proposed multidimensional static block data decomposition for simple cases of data partitioning problem on heterogeneous clusters. As shown in Figure E, first, the processes are arranged in ascending order according to their relative performance; then they are mapped into 2D node grids in row‐wise order; and finally, data are distributed over the process grid.…”

Section: Related Workmentioning

confidence: 99%

“…The optimal solution of the data partitioning problem in the case of 2‐dimensional (2D) is NP‐complete . So, the data partitioning problem on heterogeneous distributed systems is definitely NP‐complete for higher dimensions, and heuristic methods could be used to find a near‐optimal partitioning . Design of the data partitioning algorithms is done based on the performance models of the processors.…”

Section: Introductionmentioning

confidence: 99%

“…There have been few research efforts devoted to the 3D data partitioning for heterogeneous platform. Some focus on simple implementation of partitioning methods, improving load balancing and minimizing communications , . There is an inherent difficulty in optimizing both load balancing and communication simultaneously.…”

Section: Introductionmentioning

confidence: 99%

“…Some focus on simple implementation of partitioning methods, improving load balancing and minimizing communications. 7,19,30 There is an inherent difficulty in optimizing both load balancing and communication simultaneously. Load balancing optimization techniques attempt to distribute workload between all available nodes, whereas communication optimization techniques often aim to minimize the number of nodes that have worked.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

3‐D data partitioning for 3‐level perfectly nested loops on heterogeneous distributed systems

Zefreh

Lotfi

Khanli

et al. 2016

Concurrency and Computation

View full text Add to dashboard Cite

SummaryNested loops are the largest source of parallelism in many data-parallel scientific applications.Heterogeneous distributed systems are popular computing platforms for data-parallel applications. Data partitioning is critical in exploiting the computational power of such systems, and existing data partitioning algorithms try to maximize performance of data-parallel applications by finding a data distribution that balances the workload between the processing nodes while minimizing communication costs. This paper addresses the problem of 3-dimensional data partitioning for 3-level perfectly nested loops on heterogeneous distributed systems. The primary aim is to minimize the execution time by improving the load balancing and minimizing the internode communications. We propose a new data partitioning algorithm using dynamic programming, build a theoretical model to estimate the execution time of each partition, and select a partition with minimum execution time as a near-optimal solution. We demonstrate the effectiveness of the new algorithm for 2 data-parallel scientific applications on heterogeneous distributed systems. The new algorithm reduces the execution time by between 7% and 17%, on average, compared with leading data partitioning methods on 3 heterogeneous distributed systems. architecture and the need to meet ever-increasing computing needs of scientific applications, the computational capacity of the clusters is often increased. 8,9 As a first approach, we could replace all processing nodes with newer, faster ones. In this case, the cluster remains homogeneous over time, but a complete replacement of all nodes can be very costly. As a second approach, we could upgrade the cluster by adding more processing nodes that use a newer technology with higher speed. Also, we could aggregate several clusters together to use their computational power for solving computing problems. 8 Another approach is adding graphics processing units (GPUs) to improve performance of existing nodes. 9 In the latter 3 cases, the cluster becomes heterogeneous. [10][11][12] In this way, heterogeneous computing systems have emerged as an important contribution to provide computational capacity in high-performance computing. In fact, the prevalence of heterogeneous systems in the TOP500 list grew from 3.4% to 18.0%

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

3‐D data partitioning for 3‐level perfectly nested loops on heterogeneous distributed systems

Zefreh

Lotfi

Khanli

et al. 2016

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…Several studies [9,4,1] applied heuristics to arrange a set of heterogeneous processors into p × q grid and partition data accordingly with the aim to approximately balance the computing load. Another approach in [13] discussed how to find the optimal p and q in a grid that minimizes the execution time of applications.…”

Section: Figure 2 Data Partitioning Constraintsmentioning

confidence: 99%

Column-Based Partitioning for Data in High Dimensional Space

Kijsipongse

Ngamsuriyaroj

2007

2007 International Conference on Parallel Processing (ICPP 2007)

View full text Add to dashboard Cite

Several scientific applications such as 3D Jacobi Iteration [17] and LQCD [5] demand high computing power, and run on parallel systems. Such applications mostly operate on high dimensional data, and partitioning them into smaller units would help reduce their execution time considerably. Many algorithms such as CBP [2], Dissect [15], and Bisection [3] are proposed to find an optimal partitioning for two dimensional data. Simply extending such algorithms to handle higher dimensional data does not guarantee the maximum efficiency since the number of data dimensions must be taken into account. In addition, the communication cost among data in high dimensions is increased since data have high interaction to each other. This paper proposes a new algorithm called HyperCBP which is a general optimal column-based partitioning in high dimensional space. The algorithm divides high dimensional data into rectangle blocks of different sizes according to the computing power of each computing node, and minimizes the communication time used in transferring data among rectangles. We evaluate our algorithm using the new defined performance metric called Communication Saving Ratio (CSR). When compared with Dissect [15] and Bisection [3], the results show that HyperCBP gives a higher CSR than those two algorithms, and thus results in a better partitioning.

show abstract

Network‐aware optimization of communications for parallel matrix multiplication on hierarchical HPC platforms

Malik

Рычков

Lastovetsky

2015

Concurrency and Computation

View full text Add to dashboard Cite

Summary Communications on hierarchical heterogeneous high‐performance computing platforms can be optimized based on topology and performance information. For MPI, as a major programming tool for such platforms, a number of topology‐aware and performance‐aware implementations of collective operations have been proposed for optimal scheduling of messages. This approach improves performance of application and does not require to modify application source code. However, it is applicable to collective operations only and does not affect the parts of the application that are based on point‐to‐point exchanges. In this paper, we address the problem of efficient execution of data‐parallel applications on interconnected clusters and present optimizations that improve data partition by taking into account the entire communication flow of the application. This approach is also non‐intrusive to the source code but application specific. For illustration, we use parallel matrix multiplication, where the matrices are partitioned into irregular two‐dimensional rectangles assigned to different processors and arranged in columns, and the processors communicate over this partition vertically and horizontally. By rearranging the rectangles, we can minimize communications between different levels of the network hierarchy. Finding the optimal arrangement is NP‐complete; therefore, we propose two heuristic approaches based on evaluation of the communication flow on the given network topology. We demonstrate the correctness and efficiency of the proposed approaches by experimental results on multicore nodes and interconnected heterogeneous clusters. Copyright © 2015 John Wiley & Sons, Ltd.

show abstract

Multidimensional Static Block Data Decomposition for Heterogeneous Clusters

Cited by 6 publications

References 1 publication

3‐D data partitioning for 3‐level perfectly nested loops on heterogeneous distributed systems

3‐D data partitioning for 3‐level perfectly nested loops on heterogeneous distributed systems

Column-Based Partitioning for Data in High Dimensional Space

Network‐aware optimization of communications for parallel matrix multiplication on hierarchical HPC platforms

Contact Info

Product

Resources

About