Transformer model is becoming prevalent in various AI applications with its outstanding
performance. However, the high cost of computation and memory footprint make its …

Sparse-Matrix Dense-Matrix multiplication (SpMM) is the key operator for a wide range of
applications including scientific computing, graph processing, and deep learning …

This paper proposes RM-STC, a novel GPU tensor core architecture designed for sparse
Deep Neural Networks (DNNs) with two key innovations:(1) native support for both training …

Sparse linear algebra is an important kernel in many different applications. Among various
sparse general matrix-matrix multiplication (SpGEMM) algorithms, Gustavson's column-wise …

The state-of-the-art deep neural network (DNN) models use pruning to avoid over-fitting and
reduce the number of parameters. In order to improve storage and computational efficiency …

General sparse matrix-matrix multiplication (SpGEMM) is a memory-bound workload, due to
the compression format used. To minimize data movements for input matrices, outer product …

High-fan-in dot product computations are ubiquitous in highly relevant application domains,
such as signal processing and machine learning. Particularly, the diverse set of data formats …

Deep neural networks (DNNs), while enormously popular, continue to place ever higher
compute demand for which GPUs provide specialized matrix multipliers called tensor cores …

This paper examines the design space trade-offs of DNNs accelerators aiming to achieve
competitive performance and efficiency metrics for all four combinations of dense or sparse …

Recently, sparse tensor algebra (SpTA) plays an increasingly important role in machine
learning. However, due to the unstructured sparsity of SpTA, the general-purpose …