The remarkable performance of deep Convolutional neural networks (CNNs) is generally
attributed to their deeper and wider architectures, which can come with significant …

Overparameterized networks trained to convergence have shown impressive performance
in domains such as computer vision and natural language processing. Pushing state of the …

Communication systems to date primarily aim at reliably communicating bit sequences.
Such an approach provides efficient engineering designs that are agnostic to the meanings …

The growing energy and performance costs of deep learning have driven the community to
reduce the size of neural networks by selectively pruning components. Similarly to their …

The field of meta-learning, or learning-to-learn, has seen a dramatic rise in interest in recent
years. Contrary to conventional approaches to AI where tasks are solved from scratch using …

Many applications require sparse neural networks due to space or inference time
restrictions. There is a large body of work on training dense networks to yield sparse …

We rigorously evaluate three state-of-the-art techniques for inducing sparsity in deep neural
networks on two large-scale learning tasks: Transformer trained on WMT 2014 English-to …

Recently, a new trend of exploring sparsity for accelerating neural network training has
emerged, embracing the paradigm of training on the edge. This paper proposes a novel …

In this paper, we introduce a new perspective on training deep neural networks capable of
state-of-the-art performance without the need for the expensive over-parameterization by …

Recent discoveries on neural network pruning reveal that, with a carefully chosen layerwise
sparsity, a simple magnitude-based pruning achieves state-of-the-art tradeoff between …