TTOpt: A maximum volume quantized tensor train-based optimization and its application to reinforcement learning

K Sozykin, A Chertkov, R Schutski… - Advances in …, 2022 - proceedings.neurips.cc
We present a novel procedure for optimization based on the combination of efficient
quantized tensor train representation and a generalized maximum matrix volume principle …

Training scale-invariant neural networks on the sphere can happen in three regimes

M Kodryan, E Lobacheva… - Advances in Neural …, 2022 - proceedings.neurips.cc
A fundamental property of deep learning normalization techniques, such as batch
normalization, is making the pre-normalization parameters scale invariant. The intrinsic …

Spectrum Extraction and Clipping for Implicitly Linear Layers

AE Boroojeny, M Telgarsky… - … Conference on Artificial …, 2024 - proceedings.mlr.press
We show the effectiveness of automatic differentiation in efficiently and correctly computing
and controlling the spectrum of implicitly linear operators, a rich family of layer types …

Spectral regularization for adversarially-robust representation learning

S Yang, JA Zavatone-Veth, C Pehlevan - arXiv preprint arXiv:2405.17181, 2024 - arxiv.org
The vulnerability of neural network classifiers to adversarial attacks is a major obstacle to
their deployment in safety-critical applications. Regularization of network parameters during …

LOTOS: Layer-wise Orthogonalization for Training Robust Ensembles

A Ebrahimpour-Boroojeny, H Sundaram… - arXiv preprint arXiv …, 2024 - arxiv.org
Transferability of adversarial examples is a well-known property that endangers all
classification models, even those that are only accessible through black-box queries. Prior …

On the Surprising Effectiveness of Spectrum Clipping in Learning Stable Linear Dynamics

H Guo, Y Han, H Ravichandar - arXiv preprint arXiv:2412.01168, 2024 - arxiv.org
When learning stable linear dynamical systems from data, three important properties are
desirable: i) predictive accuracy, ii) provable stability, and iii) computational efficiency …

Tight and Efficient Upper Bound on Spectral Norm of Convolutional Layers

E Grishina, M Gorbunov, M Rakhuba - arXiv preprint arXiv:2409.11859, 2024 - arxiv.org
Controlling the spectral norm of the Jacobian matrix, which is related to the convolution
operation, has been shown to improve generalization, training stability and robustness in …

Spectrum Extraction and Clipping for Implicitly Linear Layers

A Ebrahimpour-Boroojeny, M Telgarsky… - … 2023 Workshop on …, 2024 - openreview.net
We show the effectiveness of automatic differentiation in efficiently and correctly computing
and controlling the spectrum of implicitly linear operators, a rich family of layer types …

Light-weight ensembling of deep neural models for object recognition in remote sensing data

I Revin, N Balabanov, A Litvintseva - Procedia Computer Science, 2023 - Elsevier
Object recognition in remote sensing is a well-developed problem in both scientific and
industrial applications. However, the limitations in computational resources often make it …