Mixed precision algorithms in numerical linear algebra
Today's floating-point arithmetic landscape is broader than ever. While scientific computing
has traditionally used single precision and double precision floating-point arithmetics, half …
has traditionally used single precision and double precision floating-point arithmetics, half …
Recovering single precision accuracy from Tensor Cores while surpassing the FP32 theoretical peak performance
Tensor Core is a mixed-precision matrix–matrix multiplication unit on NVIDIA GPUs with a
theoretical peak performance of more than 300 TFlop/s on Ampere architectures. Tensor …
theoretical peak performance of more than 300 TFlop/s on Ampere architectures. Tensor …
DASP: Specific Dense Matrix Multiply-Accumulate Units Accelerated General Sparse Matrix-Vector Multiplication
Sparse matrix-vector multiplication (SpMV) plays a key role in computational science and
engineering, graph processing, and machine learning applications. Much work on SpMV …
engineering, graph processing, and machine learning applications. Much work on SpMV …
Matrix engines for high performance computing: A paragon of performance or grasping at straws?
Matrix engines or units, in different forms and affinities, are becoming a reality in modern
processors; CPUs and otherwise. The current and dominant algorithmic approach to Deep …
processors; CPUs and otherwise. The current and dominant algorithmic approach to Deep …
Mixed precision low-rank approximations and their application to block low-rank LU factorization
We introduce a novel approach to exploit mixed precision arithmetic for low-rank
approximations. Our approach is based on the observation that singular vectors associated …
approximations. Our approach is based on the observation that singular vectors associated …
[HTML][HTML] Numerical behavior of NVIDIA tensor cores
We explore the floating-point arithmetic implemented in the NVIDIA tensor cores, which are
hardware accelerators for mixed-precision matrix multiplication available on the Volta …
hardware accelerators for mixed-precision matrix multiplication available on the Volta …
A comprehensive evaluation of novel AI accelerators for deep learning workloads
Scientific applications are increasingly adopting Artificial Intelligence (AI) techniques to
advance science. High-performance computing centers are evaluating emerging novel …
advance science. High-performance computing centers are evaluating emerging novel …
tcfft: A fast half-precision fft library for nvidia tensor cores
Mixed-precision computing becomes an inevitable trend for HPC and AI applications due to
the increasing using mixed-precision units such as NVIDIA Tensor Cores. Fast Fourier …
the increasing using mixed-precision units such as NVIDIA Tensor Cores. Fast Fourier …
Matrix multiplication in multiword arithmetic: Error analysis and application to GPU tensor cores
In multiword arithmetic, a matrix is represented as the unevaluated sum of two or more lower
precision matrices, and a matrix product is formed by multiplying the constituents in low …
precision matrices, and a matrix product is formed by multiplying the constituents in low …
DGEMM on integer matrix multiplication unit
Deep learning hardware achieves high throughput and low power consumption by reducing
computing precision and specializing in matrix multiplication. For machine learning …
computing precision and specializing in matrix multiplication. For machine learning …