It is widely acknowledged that large models have the potential to deliver superior
performance across a broad range of domains. Despite the remarkable progress made in …

Alpa automates model-parallel training of large deep learning (DL) models by generating
execution plans that unify data, operator, and pipeline parallelism. Existing model-parallel …

Large language models (LLMs) power a new generation of interactive AI applications
exemplified by ChatGPT. The interactive nature of these applications demand low job …

Deep Learning Recommender Systems (DLRSs) need to update models at low latency, thus
promptly serving new users and content. Existing DLRSs, however, fail to do so. They …

We consider the problem where $ n $ clients transmit $ d $-dimensional real-valued vectors
using $ d (1+ o (1)) $ bits each, in a manner that allows the receiver to approximately …

Deep neural networks (DNNs) have been widely adopted for various mobile inference tasks,
yet their ever-increasing computational demands are hindering their deployment on …

Data-parallel distributed training (DDT) has become the de-facto standard for accelerating
the training of most deep learning tasks on massively parallel hardware. In the DDT …

Federated learning (FL) has been recognized as a promising collaborative on-device
machine learning method in the design of Internet of Things (IoT) systems. However, most …

Gradient compression (GC) is a promising approach to addressing the communication
bottleneck in distributed deep learning (DDL). It saves the communication time, but also …

In this paper, we propose a uniformly dithered 1-bit quantization scheme for high-
dimensional statistical estimation. The scheme contains truncation, dithering, and …