Stochastic Differential Equations models for Least-Squares Stochastic Gradient Descent

Дата публикации: 17-08-2026 20:26:00

We study the dynamics of a continuous-time model of stochastic gradient descent (SGD) for the least-square problem. Indeed, pursuing the work of, we analyze stochastic differential equations (SDEs) that model SGD either in the case of the training loss (finite samples) or the population one (online setting). A key qualitative feature of the dynamics is the existence of a perfect interpolator of the data, irrespective of the sample size. In both scenarios, we provide precise, non-asymptotic rates of convergence to the (possibly degenerate) stationary distribution. Additionally, we describe this asymptotic distribution, offering estimates of its mean, deviations from it, and a proof of the emergence of heavy-tails related to the step-size magnitude. Numerical simulations supporting our findings are also presented.

Схожие новости

#	Наименование новости	Тональность	Информативность	Дата публикации
1	Asymptotics of Stochastic Gradient Descent with Dropout Regularization in Linear Models	0	5.86	17-08-2026
2	Stochastic Gradient Methods: Bias, Stability and Generalization	0	6.3	17-08-2026
3	Convergence of Decentralized Stochastic Subgradient-based Methods for Nonsmooth Nonconvex Optimization	0	8.78	17-08-2026
4	A Single-Loop Stochastic Proximal Quasi-Newton Method for Large-Scale Nonsmooth Convex Optimization	0	8	17-08-2026
5	Finite-Time Decoupled Convergence in Nonlinear Two-Time-Scale Stochastic Approximation	0	9.64	17-08-2026
6	The Distribution of Ridgeless Least Squares Interpolators	0	5.78	17-08-2026
7	High-Dimensional Analysis of Gradient Flow for Extensive-Width Quadratic Neural Networks	0	8.7	17-08-2026
8	Convergence of Noise-Free Sampling Algorithms with Regularized Wasserstein Proximals	0	8.02	17-08-2026
9	Minimax Optimal Convergence of Gradient Descent in Logistic Regression via Large and Adaptive Stepsizes	0	7.52	17-08-2026
10	A Mean-Field Analysis of Neural Stochastic Gradient Descent-Ascent for Functional Minimax Optimization	0	9.82	17-08-2026

Классификация: . Схожих патентов: 0. Схожих новостей: 10. Тональность: 0. Информативность: 6.66. Источник: jmlr.org.