Minimal Quadtree Format for Compression of Sparse Matrices Storage

imecek, Ivan; Langr, Daniel; Tvrdík, Pavel

doi:10.1109/synasc.2012.30

Cited by 8 publications

(5 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Quadtree (QT) is a tree data structure in which all inner nodes have exactly four child nodes. Since our aim is to minimize the space complexity of QT-based formats, in [7] we proposed a new QT format called minimal quadtree (MQT) format. Instead of pointers, each node of the MQT contains only 4 flags (i.e., 4 bits only) indicating whether given subquadtrees are nonempty.…”

Section: Representing Indexes In Binary Codesmentioning

confidence: 99%

“…Some recent research of hierarchical blocking SSFs, though primarily aimed at optimization of matrix-vector multiplication, also addresses optimization of memory requirements [13,14,15]. We have published several papers about space-efficient SSFs suitable for storing sparse matrices [8,11,7]. 3.…”

Section: 5mentioning

confidence: 99%

“…Within our previous work [9,12,11,8,7], we have addressed weaknesses of previously developed solutions for space-efficient formats for storing of large sparse matrices. The space complexity of the representation of sparse matrices depends strongly on the used matrix storage format.…”

Section: Introductionmentioning

confidence: 99%

“…The paper investigates memory-efficient storage formats for very large sparse matrices (LSMs). By LSMs, we mean matrices that due to their sizes must be stored and processed by massively parallel computer systems (MPCSs) with distributed memory architecture consisting of tens or hundreds of thousands of processor cores.Within our previous work [9,12,11,8,7], we have addressed weaknesses of previously developed solutions for space-efficient formats for storing of large sparse matrices. The space complexity of the representation of sparse matrices depends strongly on the used matrix storage format.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Tree-based Space Efficient Formats for Storing the Structure of Sparse Matrices

Šimeček¹,

Langr²

2014

SCPE

Self Cite

View full text Add to dashboard Cite

Abstract.Sparse storage formats describe a way how sparse matrices are stored in a computer memory. Extensive research has been conducted about these formats in the context of performance optimization of the sparse matrix-vector multiplication algorithms, but memory efficient formats for storing sparse matrices are still under development, since the commonly used storage formats (like COO or CSR) are not sufficient. In this paper, we propose and evaluate new storage formats for sparse matrices that minimize the space complexity of information about matrix structure. The first one is based on arithmetic coding and the second one is based on binary tree format. We compare the space complexity of common storage formats and our new formats and prove that the latter are considerably more space efficient.Key words: sparse matrix representation; parallel execution; space efficiency; arithmetical-coding-based format; minimal binary tree format; minimal quadtree format; AMS subject classifications. 68M14, 68W10, 68P05, 68P20, 94A171. Introduction. The paper investigates memory-efficient storage formats for very large sparse matrices (LSMs). By LSMs, we mean matrices that due to their sizes must be stored and processed by massively parallel computer systems (MPCSs) with distributed memory architecture consisting of tens or hundreds of thousands of processor cores.Within our previous work [9,12,11,8,7], we have addressed weaknesses of previously developed solutions for space-efficient formats for storing of large sparse matrices. The space complexity of the representation of sparse matrices depends strongly on the used matrix storage format. A matrix of order n is considered to be sparse if it contains much less nonzero elements than n 2 . Some alternative definitions of sparse matrix can be found in [22]. In practice, a matrix is considered sparse if the ratio of nonzero elements drops bellow some threshold. Our research addresses computations with LSMs satisfying at least one of the following conditions:1. The LSM is used repeatedly and the computation of its elements is slow and it takes more time than its later reading from a file system. 2. Construction of a LSM is memory-intensive. It needs significant amount of memory for auxiliary data structures, typically of the same order of magnitude as the amount of memory required for storing the LSM itself. 3. A solver requires the LSM in another format than is produced by a matrix generator and the conversion between these formats cannot be performed effectively on-the-fly. 4. Computational tasks with LSMs need check-pointing and recovery from failures of the MPCSs. We assume that a distributed-memory parallel computation with a LSM needs longer time. To avoid recomputations in case of a system failure, we need to save a state of these long-run processes to allow fast recovery. This is especially important nowadays (and will be more in the future) when MPCSs consist of tens or hundreds of thousands of processor cores. If at least one of these conditions is met, we might nee...

show abstract

Section: Representing Indexes In Binary Codesmentioning

confidence: 99%

Section: 5mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Tree-based Space Efficient Formats for Storing the Structure of Sparse Matrices

Šimeček¹,

Langr²

2014

SCPE

Self Cite

View full text Add to dashboard Cite

show abstract

“…These studies mainly focus on two aspects: one is reducing the complexity of the algorithm, the other is improving the efficiency by taking advantage of architectural features. As for the former aspect, Simecek et al presented a minimal Quadtree (MQT ) [7] format to compress the sparse matrices storage space. MQT stores all the nodes with an array, and each node contains four flags (i.e., 4 bits only) instead of pointers, which greatly reduces the space complexity of the quadtree storage structure.…”

Section: Introductionmentioning

confidence: 99%