Evaluating Non-square Sparse Bilinear Forms on Multiple Vector Pairs in the I/O-Model

Greiner, Gero; Jacob, Riko

doi:10.1007/978-3-642-15155-2_35

Cited by 4 publications

(6 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A lower bound for sparse N × N matrices in the I/O-model was presented in [4] and extended to non-square situations in [10]. We follow closely their description but have to restate the proof for the PEM.…”

Section: Discussionmentioning

confidence: 99%

“…Since reduce can be an arbitrary function, we restrict ourselves to matrix multiplication in a semiring, where the existence of inverse elements is not guaranteed. A lower bound for sparse N × N matrices in the I/O-model was presented in [4] and extended to non-square situations in [10]. These bounds are based on a counting argument, comparing the number of possible programs for the task with I/Os to the number of distinct matrices.…”

Section: Lower Bounds For the Shuffle Stepmentioning

confidence: 99%

“…A lower bound for sparse N × N matrices in the I/O-model was presented in [4] and extended to non-square situations in [10]. These bounds are based on a counting argument, comparing the number of possible programs for the task with I/Os to the number of distinct matrices.…”

Section: Lower Bounds For the Shuffle Stepmentioning

confidence: 99%

“…Following the model of restricting communication, the corresponding matrix is restricted to have a certain maximum number of non-zero entries per column and row. Previous work considered the multiplication of a sparse matrix with one vector [4] and several vectors simultaneously [10] in the external memory model. The I/O-complexity of transposing a dense matrix was settled in the seminal paper by Aggarval and Vitter [1] that introduced the external memory model.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

The Efficiency of MapReduce in Parallel External Memory

Greiner

Jacob

2012

LATIN 2012: Theoretical Informatics

Self Cite

View full text Add to dashboard Cite

Since its introduction in 2004, the MapReduce framework has become one of the standard approaches in massive distributed and parallel computation. In contrast to its intensive use in practise, theoretical footing is still limited and only little work has been done yet to put MapReduce on a par with the major computational models. Following pioneer work that relates the MapReduce framework with PRAM and BSP in their macroscopic structure, we focus on the functionality provided by the framework itself, considered in the parallel external memory model (PEM). In this, we present upper and lower bounds on the parallel I/O-complexity that are matching up to constant factors for the shuffle step. The shuffle step is the single communication phase where all information of one MapReduce invocation gets transferred from map workers to reduce workers. Hence, we move the focus towards the internal communication step in contrast to previous work. The results we obtain further carry over to the BSP * model. On the one hand, this shows how much complexity can be "hidden" for an algorithm expressed in MapReduce compared to PEM. On the other hand, our results bound the worst-case performance loss of the MapReduce approach in terms of I/O-efficiency.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Lower Bounds For the Shuffle Stepmentioning

confidence: 99%

Section: Lower Bounds For the Shuffle Stepmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

The Efficiency of MapReduce in Parallel External Memory

Greiner

Jacob

2012

LATIN 2012: Theoretical Informatics

Self Cite

View full text Add to dashboard Cite

show abstract

“…We will show that for certain problems and ranges of parameters, the arithmetic, communication, and storage costs of computing K k with certain Akx algorithms are within constant factors of those for computing AX via classical SpMM, indicating asymptotic optimality. Since these algorithms exploit redundant copies of inputs and intermediate quantities, it does not seem that the lower bound approaches in Bender et al (2010) and Greiner and Jacob (2010a) apply; however, it is open whether such redundancy is necessary for communication savings. We hope that these approaches, as well as those of Scquizzato and Silvestri (2014) and , will generalize and tighten the lower bounds for this problem, and motivate a more thorough exploration of the Akx design space.…”

Section: Lower Boundsmentioning

confidence: 99%

Communication lower bounds and optimal algorithms for numerical linear algebra

Ballard¹,

Carson²,

Demmel³

et al. 2014

Acta Numerica

132

View full text Add to dashboard Cite

The traditional metric for the efficiency of a numerical algorithm has been the number of arithmetic operations it performs. Technological trends have long been reducing the time to perform an arithmetic operation, so it is no longer the bottleneck in many algorithms; rather, communication, or moving data, is the bottleneck. This motivates us to seek algorithms that move as little data as possible, either between levels of a memory hierarchy or between parallel processors over a network. In this paper we summarize recent progress in three aspects of this problem. First we describe lower bounds on communication. Some of these generalize known lower bounds for dense classical (O(n3)) matrix multiplication to all direct methods of linear algebra, to sequential and parallel algorithms, and to dense and sparse matrices. We also present lower bounds for Strassen-like algorithms, and for iterative methods, in particular Krylov subspace methods applied to sparse matrices. Second, we compare these lower bounds to widely used versions of these algorithms, and note that these widely used algorithms usually communicate asymptotically more than is necessary. Third, we identify or invent new algorithms for most linear algebra problems that do attain these lower bounds, and demonstrate large speed-ups in theory and practice.

show abstract