Mathematically characterizing the implicit regularization induced by gradient-based
optimization is a longstanding pursuit in the theory of deep learning. A widespread hope is …

The loss terrains of over-parameterized neural networks have multiple global minima.
However, it is well known that stochastic gradient descent (SGD) can stably converge only to …

It is well known that (stochastic) gradient descent has an implicit bias towards flat minima. In
deep neural network training, this mechanism serves to screen out minima. However, the …

Characterizing the remarkable generalization properties of over-parameterized neural
networks remains an open problem. A growing body of recent literature shows that the bias …

A fundamental challenge at the interface of machine learning and neuroscience is to
uncover computational principles that are shared between artificial and biological neural …

As one of the most rapidly developing artificial intelligence techniques, deep learning has
been applied in various machine learning tasks and has received great attention in data …

Abstract In this thesis, Stochastic Gradient Descent (SGD), an optimization method originally
popular due to its computational efficiency, is analyzed using Markov chain methods. We …