This survey describes probabilistic algorithms for linear algebraic computations, such as
factorizing matrices and solving linear systems. It focuses on techniques that have a proven …

The asynchronous parallel implementations of stochastic gradient (SG) have been broadly
used in solving deep neural network and received many successes in practice recently …

With the fast development of deep learning, it has become common to learn big neural
networks using massive training data. Asynchronous Stochastic Gradient Descent (ASGD) is …

We describe an asynchronous parallel stochastic coordinate descent algorithm for
minimizing smooth unconstrained or separably constrained functions. The method achieves …

Finding a fixed point to a nonexpansive operator, ie, x^*=Tx^*, abstracts many problems in
numerical linear algebra, optimization, and other areas of data science. To solve fixed-point …

We introduce and analyze stochastic optimization methods where the input to each update
is perturbed by bounded noise. We show that this framework forms the basis of a unified …

We describe an asynchronous parallel stochastic proximal coordinate descent algorithm for
minimizing a composite objective function, which consists of a smooth convex function …

In recent years, mobile devices have gained increasing development with stronger
computation capability and larger storage space. Some of the computation-intensive …

Asynchronous parallel optimization received substantial successes and extensive attention
recently. One of core theoretical questions is how much speedup (or benefit) the …

We develop a family of reformulations of an arbitrary consistent linear system into a
stochastic problem. The reformulations are governed by two user-defined parameters: a …