Design and implementation of a massively parallel version of DIRECT

Abstract

Abstract. This paper describes several massively parallel implementations for a global search algorithm DIRECT. Two parallel schemes take different approaches to address DIRECT's design challenges imposed by memory requirements and data dependency. Three design aspects in topology, data structures, and task allocation are compared in detail. The goal is to analytically investigate the strengths and weaknesses of these parallel schemes, identify several key sources of inefficiency, and experimentally evaluate a number of improvements in the latest parallel DIRECT implementation. The performance studies demonstrate improved data structure efficiency and load balancing on a 2200 processor cluster.

Performance Modeling and Analysis of a Massively Parallel Direct—Part 2

“…The present work recommends that masters evaluate functions locally if the objective function cost is lower than the communication round trip cost between two nodes on the parallel system. This forms the horizontal 1-D scheme described by He et al (2006), in contrast with the vertical 1-D scheme with a single master distributing the function evaluation tasks to remote workers. Stacking function evaluations is another approach to reduce the communication overhead for distributing cheap objective function evaluations under the vertical scheme.…”

“…A local memory reduction technique-LBC (limiting box column)-was developed in He et al (2006) to take advantage of I max , which limits the number of boxes stored in memory for remaining iterations at run time, thus reducing the box memory usage. The bar plot in Figure 4.2 compares the memory allocated for holding boxes with and without LBC for all test problems.…”

“…He et al (2006) show that function evaluation tasks are distributed more efficiently to workers with a multiple subdomain search than a single domain search, especially when subdomains have unequal amounts of work. The globally shared workers have better chance to find work if the subdomain masters are not synchronized due to the data dependency in a single domain.…”

Performance Modeling and Analysis of a Massively Parallel Direct—Part 1

“…An empirical study by He et al (2006) concluded that the horizontal scheme outperforms the vertical scheme when T f ≈ 0.0, but performs worse than the vertical scheme when T f = 0.1. The performance impact of T f ∈ (0.0, 0.1) was further investigated using the 150-dimensional GR function (Appendix) under both the vertical and horizontal schemes on System X and Anantham.…”

“…Part 2 studies the impact of parallel system parameters on the performance of pDIRECT II (He et al 2006) on large-scale clusters, focusing on the latency modeling, overhead identification, and scalability analysis. The parallel system parameters include the objective function cost T f and two network characteristics T cp and T ca as shown in Table 1.1, which recounts the relevant parameters under consideration for both Part 1 and Part 2.…”