Streaming irregular arrays

Abstract

Flat data parallel array languages suffer from poor modularity. Despite being established as a high-level and expressive means of programming parallel architectures, the fact they do not support nested arrays, and that parallel functions cannot be called from within parallel contexts limits their usefulness to only a select few domains. Nested data parallel languages solve this problem, but they also assume irregularity of all nested structures. This places a cost on nesting, a cost that is needlessly paid for a certain class of programs, those where nesting is strictly regular.A second limitation of such languages is that arrays, by definition, allow for random access. This means that they must be loaded into memory in their entirety. If memory is limited, as is the case with GPUs, array languages offer no high-level means of expressing programs containing structures too large for that memory but do not require random access. This dissertation describes an extension to the Accelerate language: irregular array sequences. They allow for a limited, but still useful, form of nesting as well a streaming execution model that can work under limited memory. Furthermore, in realising irregular array sequences, we describe a generalised program flattening (vectorisation) transform that does not introduce the unnecessary cost on nesting that is incurred for strictly regular (sub)programs. This transform applies to a much broader domain than just array sequences.As a further complement to this work, this dissertation also describes two other extensions to Accelerate, a foreign function interface (FFI) and GPU-aware garbage collection. The former is the first instance of an embedded domain specific language having an FFI, and both are of considerable practical value to programmers using Accelerate framework.

Automatic Optimization of Python Skeletal Parallel Programs

“…In the second category, Accelerate [10,12] -a Haskell framework for programming GPUs -features classical data-parallel skeletons (map and variants, reduce, scan and permutation skeletons) on multi-dimensional arrays and streams. Optimization of the GPUs programs are done at runtime.…”

Position-dependent arrays and their application for high performance code generation

“…[ [3,4,5] ], [ [6,7,8,9] ] Figure 7. A three dimensional array, irregularly grouping rows of a triangular matrix.…”

“…Streaming Irregular Arrays Streaming irregular arrays [5] is an addition to the Accelerate [13] language which provides support for irregular data structures backed by a flattened representation, and provides type-system support for reasoning with nested irregular arrays. Unlike the work presented here however, it relies on run-time support for tracking the sizes of arrays, as opposed to attempting to resolve index computations fully at compile time.…”