This work studies post-training parameter quantization in large language models (LLMs).
We introduce quantization with incoherence processing (QuIP), a new method based on the …

Symmetries and equivariance are fundamental to the generalization of neural networks on
domains such as images, graphs, and point clouds. Existing work has primarily focused on a …

For many modern applications in science and engineering, data are collected in a streaming
fashion carrying time-varying information, and practitioners need to process them with a …

Recent advances in efficient Transformers have exploited either the sparsity or low-rank
properties of attention matrices to reduce the computational and memory bottlenecks of …

Distributed model training suffers from communication overheads due to frequent gradient
updates transmitted between compute nodes. To mitigate these overheads, several studies …

Matrix completion is a basic machine learning problem that has wide applications,
especially in collaborative filtering and recommender systems. Simple non-convex …

Can we recover a complex signal from its Fourier magnitudes? More generally, given a set
of m measurements, y_k=\left| a _k^* x\right| yk= ak∗ x for k= 1, ..., mk= 1,…, m, is it possible …

We show that there are no spurious local minima in the non-convex factorized
parametrization of low-rank matrix recovery from incoherent linear measurements. With …

Matrix factorization is a popular approach for large-scale matrix completion. The optimization
formulation based on matrix factorization, even with huge size, can be solved very efficiently …

In this paper we study the problem of recovering a low-rank matrix from linear
measurements. Our algorithm, which we call Procrustes Flow, starts from an initial estimate …