A Survey of Numerical Methods Utilizing Mixed Precision Arithmetic

Abdelfattah, Ahmad; Anzt, Hartwig; Boman, Erik G.; Carson, Erin; Cojean, Terry; Dongarra, Jack; Gates, Mark; Grützmacher, Thomas; Higham, Nicholas J.; Li, Sherry; Lindquist, Neil; Liu, Yang; Loe, Jennifer; Łuszczek, Piotr; Nayak, Pratik; Pranesh, Sri; Rajamanickam, S; Ribizel, Tobias; Smith, Barry; Świrydowicz, Kasia; Thomas, Stephen; Tomov, Stanimire; Tsai, Yaohung M.; Yamazaki, Ichitaro; Yang, Urike Meier

doi:10.48550/arxiv.2007.06674

Cited by 12 publications

(20 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, this may well change: first, when high accuracy is needed, n may need to be large enough to enter the regime E N E D . Second, and more nontrivially, the breakeven point where E D ≈ E N = O( κ(A)) depends on the working precision , and a compelling line of recent research is to use low-precision arithmetic for efficiency [1] in scientific computing and data science applications. In such situation, E N would start dominating for a modest discretization size, making FOP an important technique to retain good solutions.…”

Section: Mathematical Basics 21 Numerical Solution Of Operator Equationsmentioning

confidence: 99%

Full operator preconditioning and the accuracy of solving linear systems

Mohr,

Nakatsukasa,

Urzúa-Torres

2021

Preprint

View full text Add to dashboard Cite

Unless special conditions apply, the attempt to solve ill-conditioned systems of linear equations with standard numerical methods leads to uncontrollably high numerical error. Often, such systems arise from the discretization of operator equations with a large number of discrete variables. In this paper we show that the accuracy can be improved significantly if the equation is transformed before discretization, a process we call full operator preconditioning (FOP). It bears many similarities with traditional preconditioning for iterative methods but, crucially, transformations are applied at the operator level. We show that while condition-number improvements from traditional preconditioning generally do not improve the accuracy of the solution, FOP can. A number of topics in numerical analysis can be interpreted as implicitly employing FOP; we highlight (i) Chebyshev interpolation in polynomial approximation, and (ii) Olver-Townsend's spectral method, both of which produce solutions of dramatically improved accuracy over a naive problem formulation. In addition, we propose a FOP preconditioner based on integration for the solution of fourth-order differential equations with the finite-element method, showing the resulting linear system is well-conditioned regardless of the discretization size, and demonstrate its error-reduction capabilities on several examples. This work shows that FOP can improve accuracy beyond the standard limit for both direct and iterative methods.

show abstract

Section: Mathematical Basics 21 Numerical Solution Of Operator Equationsmentioning

confidence: 99%

Full operator preconditioning and the accuracy of solving linear systems

Mohr,

Nakatsukasa,

Urzúa-Torres

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…While modern GPUs provide higher memory bandwidth than FPGA accelerator cards (from 549 GB/s of the Nvidia P100 to the 1555 GB/s of the Nvidia A100), existing SpMV implementations are often unable to utilize the available bandwidth fully [11]. Moreover, optimizations for reduced precision data-types are currently limited to half-precision floating-point, with no support for reduced-precision fixed-point arithmetic [12], [13] although recent work has explored heuristics to mix single and double precision floating-point arithmetic [14].…”

Section: Related Workmentioning

confidence: 99%

Scaling up HBM Efficiency of Top-K SpMV for Approximate Embedding Similarity on FPGAs

Parravicini

Cellamare

Siracusa

et al. 2021

2021 58th ACM/IEEE Design Automation Conference (DAC)

View full text Add to dashboard Cite

Top-K SpMV is a key component of similaritysearch on sparse embeddings. This sparse workload does not perform well on general-purpose NUMA systems that employ traditional caching strategies. Instead, modern FPGA accelerator cards have a few tricks up their sleeve. We introduce a Top-K SpMV FPGA design that leverages reduced precision and a novel packet-wise CSR matrix compression, enabling custom data layouts and delivering bandwidth efficiency often unreachable even in architectures with higher peak bandwidth. With HBMbased boards, we are 100x faster than a multi-threaded CPU implementation and 2x faster than a GPU with 20% higher bandwidth, with 14.2x higher power-efficiency.

show abstract

“…The computational cost of the function evaluations F (y (j) ) can be considerable, especially in cases where it necessitates an implicit solve. The use of mixed precision approach has been implemented for other numerical methods [1,5] seems to be a promising approach. Lowering the precision on these computations, either by storing F (y (j) ) as a single precision variable rather than a double precision one, or by raising the tolerance of the implicit solver, can speed up the computation significantly.…”

Section: Introduction Consider the Ordinary Differential Equation (Ode)mentioning

confidence: 99%

Perturbed Runge-Kutta methods for mixed precision applications

Grant¹

2020

Preprint

View full text Add to dashboard Cite

A Survey of Numerical Methods Utilizing Mixed Precision Arithmetic

Cited by 12 publications

References 0 publications

Full operator preconditioning and the accuracy of solving linear systems

Full operator preconditioning and the accuracy of solving linear systems

Scaling up HBM Efficiency of Top-K SpMV for Approximate Embedding Similarity on FPGAs

Perturbed Runge-Kutta methods for mixed precision applications

Contact Info

Product

Resources

About