Architecture of an automatically tuned linear algebra library

Processes Distribution of Homogeneous Parallel Linear Algebra Routines on Heterogeneous Clusters

“…In this paper, we have presented an extension of our previous self-optimization methodology for homogeneous systems [14], where several decisions (algorithmic parameters) were taken automatically in order to obtain execution times close to the optimum with parallel linear algebra routines. This technique has been combined with our approach of work distribution for parallel dynamic programming [8].…”

“…In [14] we presented a self-optimization methodology for these routines on homogeneous systems (it was a HoHo strategy), where several decisions (algorithmic parameters) were taken automatically in order to obtain execution times close to the optimum, and without rewriting any code line of the routines. Some of these decisions are: number of processes to generate, which processors are used (if the system workload is heterogeneous, it could be interesting not to use the overloaded processors [15]), the logical topology of processes (normally, in parallel linear algebra routines the processes are organized in a logical 2D mesh to distribute the data and to perform the inter-processor communications) and the block size for both data distribution and computation.…”

“…The starting point of this work consists of the proposed model for homogeneous parallel system with P processors of similar physical characteristics, homogeneously connected [14,16]. This model is constituted by a set of parameters that characterize the system (system parameters, SP ), divided in two groups: arithmetic system parameters (SP ari ) and communication system parameters (SP com …”

Empirical Installation of Linear Algebra Shared-Memory Subroutines for Auto-Tuning

“…The goal is to select the most appropriate number of threads at each level of parallelism, together with the values of other algorithmic parameters, like the block size (or sizes) in algorithms by blocks. The methodology of [9] is adapted to the empirical installation in NUMA systems. It is divided in three phases, which are represented in figure 1: -Design phase.…”

“…The libraries are optimized in the installation process for shared memory machines [29] or for message-passing systems [8,9]. The routines can adapt to the conditions of the system at a particular moment [23].…”