Organizing matrices and matrix operations for paged memory systems

McKellar, A. C.; Coffman, E. G.

doi:10.1145/362875.362879

Cited by 152 publications

(84 citation statements)

References 8 publications

(7 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The technique of computing the result of a matrix multiplication by tiling the output by squares is very old indeed [16]. In the map-reduce model, that is correct if a single round of map-reduce is used, but, as we shall see in Section 6.3, not quite correct for two-phase matrix multiplication, where the minimum cost occurs when the matrices are tiled with rectangles of aspect ratio 2:1.…”

Section: Matching Upper Bound On Replication Ratementioning

confidence: 99%

Upper and lower bounds on the cost of a map-reduce computation

et al. 2013

View full text Add to dashboard Cite

In this paper we study the tradeoff between parallelism and communication cost in a map-reduce computation. For any problem that is not "embarrassingly parallel," the finer we partition the work of the reducers so that more parallelism can be extracted, the greater will be the total communication between mappers and reducers. We introduce a model of problems that can be solved in a single round of mapreduce computation. This model enables a generic recipe for discovering lower bounds on communication cost as a function of the maximum number of inputs that can be assigned to one reducer. We use the model to analyze the tradeoff for three problems: finding pairs of strings at Hamming distance d, finding triangles and other patterns in a larger graph, and matrix multiplication. For finding strings of Hamming distance 1, we have upper and lower bounds that match exactly. For triangles and many other graphs, we have upper and lower bounds that are the same to within a constant factor. For the problem of matrix multiplication, we have matching upper and lower bounds for one-round map-reduce algorithms. We are also able to explore tworound map-reduce algorithms for matrix multiplication and show that these never have more communication, for a given reducer size, than the best one-round algorithm, and often have significantly less.

show abstract

Section: Matching Upper Bound On Replication Ratementioning

confidence: 99%

Upper and lower bounds on the cost of a map-reduce computation

et al. 2013

View full text Add to dashboard Cite

show abstract

“…Tiling has been extensively studied to improve performance of scientific and engineering codes [2,11,13,17] for parallel execution [16] and as a mechanism to improve locality [17]. However, most programming languages do not provide any support for tiles.…”

Section: Related Workmentioning

confidence: 99%

A Parallel Numerical Solver Using Hierarchically Tiled Arrays

Brodman

Evans

Manguoğlu³

et al. 2011

Languages and Compilers for Parallel Computing

View full text Add to dashboard Cite

Abstract. Solving linear systems is an important problem for scientific computing. Exploiting parallelism is essential for solving complex systems, and this traditionally involves writing parallel algorithms on top of a library such as MPI. The SPIKE family of algorithms is one well-known example of a parallel solver for linear systems. The Hierarchically Tiled Array data type extends traditional data-parallel array operations with explicit tiling and allows programmers to directly manipulate tiles. The tiles of the HTA data type map naturally to the block nature of many numeric computations, including the SPIKE family of algorithms. The higher level of abstraction of the HTA enables the same program to be portable across different platforms. Current implementations target both shared-memory and distributed-memory models. In this paper we present a proof-of-concept for portable linear solvers. We implement two algorithms from the SPIKE family using the HTA library. We show that our implementations of SPIKE exploit the abstractions provided by the HTA to produce a compact, clean code that can run on both shared-memory and distributedmemory models without modification. We discuss how we map the algorithms to HTA programs as well as examine their performance. We compare the performance of our HTA codes to comparable codes written in MPI as well as current state-of-the-art linear algebra routines.

show abstract

“…The idea of chunking multi-dimensional arrays has its origins from techniques used in scientific computing for managing memory resident sparse matrices [2] and large sparse and dense matrices in paged and parallel environments [10,11]. We illustrate an addressing method for array chunks with a technique for sparse multi-dimensional arrays.…”

Section: Addressing Array Chunksmentioning

confidence: 99%

Chunking of Large Multidimensional Arrays

Rotem¹,

Otoo²,

Seshadri³

2007

View full text Add to dashboard Cite

Very large multidimensional arrays are commonly used in data intensive scientific computations as well as on-line analytical processing applications referred to as MOLAP. The storage organization of such arrays on disks is done by partitioning the large global array into fixed size sub-arrays called chunks or tiles that form the units of data transfer between disk and memory. Typical queries involve the retrieval of sub-arrays in a manner that accesses all chunks that overlap the query results. An important metric of the storage efficiency is the expected number of chunks retrieved over all such queries. The question that immediately arises is "what shapes of array chunks give the minimum expected number of chunks over a query workload?" The problem of optimal chunking was first introduced by Sarawagi and Stonebraker [14] who gave an approximate solution. In this paper we develop exact mathematical models of the problem and provide exact solutions using steepest descent and geometric programming methods. Experimental results, using synthetic and real life workloads, show that our solutions are consistently less than 2.0% of the true number of chunks retrieved for any number of dimensions. In contrast, the approximate solution of [14] can deviate considerably from the true result with increasing number of dimensions.

show abstract

Organizing matrices and matrix operations for paged memory systems

Cited by 152 publications

References 8 publications

Upper and lower bounds on the cost of a map-reduce computation

Upper and lower bounds on the cost of a map-reduce computation

A Parallel Numerical Solver Using Hierarchically Tiled Arrays

Chunking of Large Multidimensional Arrays

Contact Info

Product

Resources

About