Wearable devices are a fast-growing technology with impact on personal healthcare for both
society and economy. Due to the widespread of sensors in pervasive and distributed …

Distributed deep learning (DL) has become prevalent in recent years to reduce training time
by leveraging multiple computing devices (eg, GPUs/TPUs) due to larger models and …

The growing energy and performance costs of deep learning have driven the community to
reduce the size of neural networks by selectively pruning components. Similarly to their …

Many applications require sparse neural networks due to space or inference time
restrictions. There is a large body of work on training dense networks to yield sparse …

We rigorously evaluate three state-of-the-art techniques for inducing sparsity in deep neural
networks on two large-scale learning tasks: Transformer trained on WMT 2014 English-to …

As neural network model sizes have dramatically increased, so has the interest in various
techniques to reduce their parameter counts and accelerate their execution. An active area …

Since hardware resources are limited, the objective of training deep learning models is
typically to maximize accuracy subject to the time and memory constraints of training and …

We introduce a new method for scaling up distributed Stochastic Gradient Descent (SGD)
training of Deep Neural Networks (DNN). The method solves the well-known communication …

Sparse neural networks are becoming increasingly important as the field seeks to improve
the performance of existing models by scaling them up, while simultaneously trying to …

The training of sparse neural networks is becoming an increasingly important tool for
reducing the computational footprint of models at training and evaluation, as well enabling …