The training and generalization dynamics of the Transformer's core mechanism, namely the
Attention mechanism, remain under-explored. Besides, existing analyses primarily focus on …

Bayesian models are powerful tools for studying complex data, allowing the analyst to
encode rich hierarchical dependencies and leverage prior information. Most importantly …

Concurrent multiscale structural optimization is concerned with the improvement of
macroscale structural performance through the design of microscale architectures. The …

While significant theoretical progress has been achieved, unveiling the generalization
mystery of overparameterized neural networks still remains largely elusive. In this paper, we …

This paper explores the connection between learning trajectories of Deep Neural Networks
(DNNs) and their generalization capabilities when optimized using (stochastic) gradient …

Bayesian models are a powerful tool for studying complex data, allowing the analyst to
encode rich hierarchical dependencies and leverage prior information. Most importantly …

We give sharper bounds for uniformly stable randomized algorithms in a PAC-Bayesian
framework, which improve the existing results by up to a factor of $\sqrt {n} $(ignoring a log …

This paper explores the generalization characteristics of iterative learning algorithms with
bounded updates for non-convex loss functions, employing information-theoretic …

Decentralized Stochastic Gradient Descent (D-SGD) represents an efficient communication
approach tailored for mastering insights from vast, distributed datasets. Inspired by parallel …

Generalization analysis of learning algorithms often builds on a critical assumption that
training examples are independently and identically distributed, which is often violated in …