Extending a hierarchical tiling arrays library to support sparse data partitioning

IEEE Trans. Parallel Distrib. Syst.

Torres

Fresno

et al. 2014

Self Cite

Abstract-Automatic data distribution is a key feature to obtain efficient implementations from abstract and portable parallel codes. We present a highly efficient and extensible runtime library that integrates techniques for automatic data partition and mapping. It uses a novel approach to define an abstract interface and a plug-in system to encapsulate different types of regular and irregular techniques, helping to generate codes which are independent of the exact mapping functions selected. Currently, it supports hierarchical tiling of arrays with dense and stride domains, that allows the implementation of both data and task parallelism using a SPMD model. It automatically computes appropriate domain partitions for a selected virtual topology, mapping them to available processors with static or dynamic load-balancing techniques. Our library also allows the construction of reusable communication patterns that efficiently exploit MPI communication capabilities. The use of our library greatly reduces the complexity of data distribution and communication, hiding the details of the underlying architecture. The library can be used as an abstract layer for building generic tiling operations as well. Our experimental results show that the use of this library allows to achieve similar performance as carefully-implemented manual versions for several, well-known parallel kernels and benchmarks in distributed and multicore systems, and substantially reduces programming effort.

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

An Extensible System for Multilevel Automatic Data Partition and Mapping

IEEE Trans. Parallel Distrib. Syst.

Torres

Fresno

et al. 2014

Self Cite

“…It has also been extended to support sparse data structures such as sparse matrices, or graphs [2], using the same methodology and interfaces. Hitmap is based on a distributed SPMD programming model, using (n,S0,S1,...,Sn) abstractions to declare data structures with a global view.…”

Section: Hitmap In a Nutshellmentioning

confidence: 99%

“…Hitmap reduces code development effort performing highly-efficient data distributions and aggregated communications, expressed in an abstract form. It can be used for indexed dense or sparse data structures, such as arrays, or graphs [2]. The mapping decisions are automatically guided by topology, partition, and distribution policies encapsulated in plug-in modules.…”

Section: Introductionmentioning

confidence: 99%

Exploiting distributed and shared memory hierarchies with Hitmap

Moreton-Fernandez

2014 International Conference on High Performance Computing &Amp; Simulation (HPCS)

Llanos

2014

Self Cite

Current multicomputers are typically built as interconnected clusters of shared-memory multicore computers. A common programming approach for these clusters is to simply use a message-passing paradigm, launching as many processes as cores available. Nevertheless, to better exploit the scalability of these clusters and highly-parallel multicore systems, it is needed to efficiently use their distributed-and shared-memory hierarchies. This implies to combine different programming paradigms and tools at different levels of the program design.This paper presents an approach to ease the programming for mixed distributed and shared memory parallel computers. The coordination at the distributed memory level is simplified using Hitmap, a library for distributed computing using hierarchical tiling of data structures. We show how this tool can be integrated with shared-memory programming models and automatic code generation tools to efficiently exploit the multicore environment of each multicomputer node. This approach allows to exploit the most appropriate techniques for each model, easily generating multilevel parallel programs that automatically adapt their communication and synchronization structures to the target machine. Our experimental results show how this approach mimics or even improves the best performance results obtained with manually optimized codes using pure MPI or OpenMP models.

“…Hitmap [3], [13] is a library for hierarchical tiling and mapping of dense arrays. It is based on a distributed SPMD programming model, using abstractions to declare data structures with a global view, and automatizes the partition, mapping, and communication of hierarchies of tiles, while still delivering good performance.…”

Section: The Original Hitmap Librarymentioning

confidence: 99%

Blending Extensibility and Performance in Dense and Sparse Parallel Data Management

Fresno

IEEE Trans. Parallel Distrib. Syst.

Llanos

2014

Self Cite

Dealing with both dense and sparse data in parallel environments usually leads to two different approaches: To rely on a monolithic, hard-to-modify parallel library, or to code all data management details by hand. In this paper we propose a third approach, that delivers good performance while the underlying library structure remains modular and extensible. Our solution integrates dense and sparse data management using a common interface, that also decouples data representation, partitioning, and layout from the algorithmic and parallel strategy decisions of the programmer. Our experimental results in different parallel environments show that this new approach combines the flexibility obtained when the programmer handles all the details with a performance comparable to the use of a state-of-the-art, sparse matrix parallel library.