Low Rank Approximation of Multidimensional Data

Azaïez, Mejdi; Lestandi, Lucas; Rebollo, Tomás Chacón

doi:10.1007/978-3-030-17012-7_5

Cited by 6 publications

(4 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similar to this work, the authors of [44] present matrix decomposition methods as an effective compression technique for large-scale simulation data, though pass-efficiency is not emphasized in their work. In [45] and [46], the authors present online methods for maintaining a low-rank SVD approximation of simulation data via rank-one updates; this procedure requires significant computation at each step, however.…”

Section: Introductionmentioning

confidence: 90%

Pass-efficient methods for compression of high-dimensional turbulent flow data

Dunton

Jofre

Iaccarino

et al. 2020

Journal of Computational Physics

View full text Add to dashboard Cite

Section: Introductionmentioning

confidence: 90%

Pass-efficient methods for compression of high-dimensional turbulent flow data

Dunton

Jofre

Iaccarino

et al. 2020

Journal of Computational Physics

View full text Add to dashboard Cite

“…Low-rank matrix methods have been used for the compression of large-scale simulation data in works such as [35]. Computing low-rank matrix approximations in which simulation data arrives into working memory online has been addressed in [36,37,38].…”

Section: Contribution Of This Workmentioning

confidence: 99%

Task-parallel in-situ temporal compression of large-scale computational fluid dynamics data

Pacella¹,

Dunton²,

Doostan³

et al. 2021

Preprint

View full text Add to dashboard Cite

Present day computational fluid dynamics (CFD) simulations generate extremely large amounts of data, sometimes on the order of TB/s. Often, a significant fraction of this data is discarded because current storage systems are unable to keep pace. To address this, data compression algorithms can be applied to data arrays containing flow quantities of interest (QoIs) to reduce the overall amount of storage. Compression methods either exactly reconstruct the original dataset (lossless compression) or provide an approximate representation of the original dataset (lossy compression). The matrix column interpolative decomposition (ID) can be implemented as a type of lossy compression for data matrices that factors the original data matrix into a product of two smaller factor matrices. One of these matrices consists of a subset of the columns of the original data matrix, while the other is a coefficient matrix which approximates the columns of the original data matrix as linear combinations of the selected columns. Motivating this work is the observation that the structure of ID algorithms makes them a natural fit for the asynchronous nature of task-based parallelism; they are able to operate independently on sub-domains of the system of interest and, as a result, provide varied levels of compression. Using the task-based Legion programming model, a single-pass ID algorithm (SPID) for CFD applications is implemented. Performance studies, scalability, and the accuracy of the compression algorithms are presented for a benchmark analytical Taylor-Green vortex problem, followed by a large-scale implementation of a compressible Taylor-Green vortex using a high-order Navier-Stokes solver. In both cases, compression factors exceeding 100 are achieved with relative errors at or below 10 −3 . Moreover, strong and weak scaling results demonstrate that introducing SPID to solvers leads to negligible increases in runtime.

show abstract

“…Low-rank matrix methods have been used for the compression of large-scale simulation data in works such as Azaïez et al (2019). Computing low-rank matrix approximations in which simulation data arrives into working memory online has been addressed in Brand (2006), Zimmermann et al (2018), and Tropp et al (2019). The present effort is focused on using low-rank matrix approximation to build in situ compression methods which exploit task-based parallelism to achieve high compression and concurrency on heterogeneous modern computing architectures.…”

Section: Introductionmentioning

confidence: 99%

Task-parallel in situ temporal compression of large-scale computational fluid dynamics data

Pacella

Dunton

Doostan

et al. 2022

The International Journal of High Performance Computing Applica

View full text Add to dashboard Cite

Present day computational fluid dynamics (CFD) simulations generate considerable amounts of data, sometimes on the order of TB/s. Often, a significant fraction of this data is discarded because current storage systems are unable to keep pace. To address this, data compression algorithms can be applied to data arrays containing flow quantities of interest (QoIs) to reduce the overall required storage. The matrix column interpolative decomposition (ID) can be implemented as a type of lossy compression for data matrices that factors the original data matrix into a product of two smaller factor matrices. One of these matrices consists of a subset of the columns of the original data matrix, while the other is a coefficient matrix which approximates the original data matrix columns as linear combinations of the selected columns. Motivating this work is the observation that the structure of ID algorithms makes them well suited for the asynchronous nature of task-based parallelism; they can operate independently on subdomains of the system of interest and, as a result, provide varied levels of compression. Using the task-based Legion programming model, a single-pass ID algorithm (SPID) for CFD applications is implemented. Performance studies, scalability, and the accuracy of the compression algorithm are presented for a benchmark analytical Taylor-Green vortex problem, as well as large-scale implementations of both low and high Reynolds number ( Re) compressible Taylor-Green vortices using a high-order Navier-Stokes solver. In the case of the analytical solution, the resulting compressed solution was rank-one, with error on the order of machine precision. For the low- Re vortex, compression factors between 1000 and 10,000 were achieved for errors in the range 10−2–10−3. Similar error values were seen for the high- Re vortex, this time with compression factors between 100 and 1000. Moreover, strong and weak scaling results demonstrate that introducing SPID to solvers leads to negligible increases in runtime.

show abstract

Low Rank Approximation of Multidimensional Data

Cited by 6 publications

References 48 publications

Pass-efficient methods for compression of high-dimensional turbulent flow data

Pass-efficient methods for compression of high-dimensional turbulent flow data

Task-parallel in-situ temporal compression of large-scale computational fluid dynamics data

Task-parallel in situ temporal compression of large-scale computational fluid dynamics data

Contact Info

Product

Resources

About