TTOpt: A maximum volume quantized tensor train-based optimization and its application to reinforcement learning
We present a novel procedure for optimization based on the combination of efficient
quantized tensor train representation and a generalized maximum matrix volume principle …
quantized tensor train representation and a generalized maximum matrix volume principle …
Training scale-invariant neural networks on the sphere can happen in three regimes
M Kodryan, E Lobacheva… - Advances in Neural …, 2022 - proceedings.neurips.cc
A fundamental property of deep learning normalization techniques, such as batch
normalization, is making the pre-normalization parameters scale invariant. The intrinsic …
normalization, is making the pre-normalization parameters scale invariant. The intrinsic …
Spectrum Extraction and Clipping for Implicitly Linear Layers
AE Boroojeny, M Telgarsky… - … Conference on Artificial …, 2024 - proceedings.mlr.press
We show the effectiveness of automatic differentiation in efficiently and correctly computing
and controlling the spectrum of implicitly linear operators, a rich family of layer types …
and controlling the spectrum of implicitly linear operators, a rich family of layer types …
Spectral regularization for adversarially-robust representation learning
S Yang, JA Zavatone-Veth, C Pehlevan - arXiv preprint arXiv:2405.17181, 2024 - arxiv.org
The vulnerability of neural network classifiers to adversarial attacks is a major obstacle to
their deployment in safety-critical applications. Regularization of network parameters during …
their deployment in safety-critical applications. Regularization of network parameters during …
LOTOS: Layer-wise Orthogonalization for Training Robust Ensembles
A Ebrahimpour-Boroojeny, H Sundaram… - arXiv preprint arXiv …, 2024 - arxiv.org
Transferability of adversarial examples is a well-known property that endangers all
classification models, even those that are only accessible through black-box queries. Prior …
classification models, even those that are only accessible through black-box queries. Prior …
On the Surprising Effectiveness of Spectrum Clipping in Learning Stable Linear Dynamics
H Guo, Y Han, H Ravichandar - arXiv preprint arXiv:2412.01168, 2024 - arxiv.org
When learning stable linear dynamical systems from data, three important properties are
desirable: i) predictive accuracy, ii) provable stability, and iii) computational efficiency …
desirable: i) predictive accuracy, ii) provable stability, and iii) computational efficiency …
Tight and Efficient Upper Bound on Spectral Norm of Convolutional Layers
E Grishina, M Gorbunov, M Rakhuba - arXiv preprint arXiv:2409.11859, 2024 - arxiv.org
Controlling the spectral norm of the Jacobian matrix, which is related to the convolution
operation, has been shown to improve generalization, training stability and robustness in …
operation, has been shown to improve generalization, training stability and robustness in …
Spectrum Extraction and Clipping for Implicitly Linear Layers
A Ebrahimpour-Boroojeny, M Telgarsky… - … 2023 Workshop on …, 2024 - openreview.net
We show the effectiveness of automatic differentiation in efficiently and correctly computing
and controlling the spectrum of implicitly linear operators, a rich family of layer types …
and controlling the spectrum of implicitly linear operators, a rich family of layer types …
Light-weight ensembling of deep neural models for object recognition in remote sensing data
I Revin, N Balabanov, A Litvintseva - Procedia Computer Science, 2023 - Elsevier
Object recognition in remote sensing is a well-developed problem in both scientific and
industrial applications. However, the limitations in computational resources often make it …
industrial applications. However, the limitations in computational resources often make it …