Stochastic gradient descent with noise of machine learning type part i: Discrete time analysis

S Wojtowytsch - Journal of Nonlinear Science, 2023 - Springer
Stochastic gradient descent (SGD) is one of the most popular algorithms in modern machine
learning. The noise encountered in these applications is different from that in many …

Deep learning for limit order books

JA Sirignano - Quantitative Finance, 2019 - Taylor & Francis
This paper develops a new neural network architecture for modeling spatial distributions (ie
distributions on R d) which is more computationally efficient than a traditional fully …

Fast mixing of stochastic gradient descent with normalization and weight decay

Z Li, T Wang, D Yu - Advances in Neural Information …, 2022 - proceedings.neurips.cc
Abstract We prove the Fast Equilibrium Conjecture proposed by Li et al.,(2020), ie,
stochastic gradient descent (SGD) on a scale-invariant loss (eg, using networks with various …

Deep learning volatility

B Horvath, A Muguruza, M Tomas - arXiv preprint arXiv:1901.09647, 2019 - arxiv.org
We present a neural network based calibration method that performs the calibration task
within a few milliseconds for the full implied volatility surface. The framework is consistently …

[图书][B] Parameter estimation in stochastic volatility models

JPN Bishwal - 2022 - Springer
In this book, we study stochastic volatility models and methods of pricing, hedging, and
estimation. Among models, we will study models with heavy tails and long memory or long …

Law of balance and stationary distribution of stochastic gradient descent

L Ziyin, H Li, M Ueda - arXiv preprint arXiv:2308.06671, 2023 - arxiv.org
The stochastic gradient descent (SGD) algorithm is the algorithm we use to train neural
networks. However, it remains poorly understood how the SGD navigates the highly …

Analysis of stochastic gradient descent in continuous time

J Latz - Statistics and Computing, 2021 - Springer
Stochastic gradient descent is an optimisation method that combines classical gradient
descent with random subsampling within the target functional. In this work, we introduce the …

Strong error analysis for stochastic gradient descent optimization algorithms

A Jentzen, B Kuckuck, A Neufeld… - IMA Journal of …, 2021 - academic.oup.com
Stochastic gradient descent (SGD) optimization algorithms are key ingredients in a series of
machine learning applications. In this article we perform a rigorous strong error analysis for …

Stationary behavior of constant stepsize SGD type algorithms: An asymptotic characterization

Z Chen, S Mou, ST Maguluri - Proceedings of the ACM on Measurement …, 2022 - dl.acm.org
Stochastic approximation (SA) and stochastic gradient descent (SGD) algorithms are work-
horses for modern machine learning algorithms. Their constant stepsize variants are …

Uniform-in-time weak error analysis for stochastic gradient descent algorithms via diffusion approximation

Y Feng, T Gao, L Li, JG Liu, Y Lu - arXiv preprint arXiv:1902.00635, 2019 - arxiv.org
Diffusion approximation provides weak approximation for stochastic gradient descent
algorithms in a finite time horizon. In this paper, we introduce new tools motivated by the …