Domain-specific hardware is becoming a promising topic in the backdrop of improvement
slow down for general-purpose processors due to the foreseeable end of Moore's Law …

Recurrent Neural Networks are ubiquitous and pervasive in many artificial intelligence
applications such as speech recognition, predictive healthcare, creative art, and so on …

High network communication cost for synchronizing gradients and parameters is the well-
known bottleneck of distributed training. In this work, we propose TernGrad that uses ternary …

Large Language Models (LLMs), renowned for their remarkable performance across diverse
domains, present a challenge when it comes to practical deployment due to their colossal …

Large language models have recently achieved state of the art performance across a wide
variety of natural language tasks. Meanwhile, the size of these models and their latency …

Channel pruning is an important method to speed up CNN model's inference. Previous filter
pruning algorithms regard importance evaluation and model fine-tuning as two independent …

This paper presents a complete solution for the hardware design of a memristor-based long
short-term memory (MLSTM) network. Throughout the design process, we fully consider the …

We're releasing highly optimized GPU kernels for an underexplored class of neural network
architectures: networks with block-sparse weights. The kernels allow for efficient evaluation …

In recent years, researchers tend to pre-train ever-larger language models to explore the
upper limit of deep models. However, large language model pre-training costs intensive …

In seeking for sparse and efficient neural network models, many previous works investigated
on enforcing L1 or L0 regularizers to encourage weight sparsity during training. The L0 …