Annihilation of spurious minima in two-layer relu networks
Y Arjevani, M Field - Advances in Neural Information …, 2022 - proceedings.neurips.cc
We study the optimization problem associated with fitting two-layer ReLU neural networks
with respect to the squared loss, where labels are generated by a target network. Use is …
with respect to the squared loss, where labels are generated by a target network. Use is …
Analytic study of families of spurious minima in two-layer relu neural networks: a tale of symmetry ii
Y Arjevani, M Field - Advances in Neural Information …, 2021 - proceedings.neurips.cc
We study the optimization problem associated with fitting two-layer ReLU neural networks
with respect to the squared loss, where labels are generated by a target network. We make …
with respect to the squared loss, where labels are generated by a target network. We make …
Symmetry & critical points for a model shallow neural network
Y Arjevani, M Field - Physica D: Nonlinear Phenomena, 2021 - Elsevier
Using methods based on the analysis of real analytic functions, symmetry and equivariant
bifurcation theory, we obtain sharp results on families of critical points of spurious minima …
bifurcation theory, we obtain sharp results on families of critical points of spurious minima …
Symmetry & critical points for symmetric tensor decompositions problems
Y Arjevani, G Vinograd - arXiv preprint arXiv:2306.07886, 2023 - arxiv.org
We consider the non-convex optimization problem associated with the decomposition of a
real symmetric tensor into a sum of rank one terms. Use is made of the rich symmetry …
real symmetric tensor into a sum of rank one terms. Use is made of the rich symmetry …
Equivariant bifurcation, quadratic equivariants, and symmetry breaking for the standard representation of sk
Y Arjevani, M Field - Nonlinearity, 2022 - iopscience.iop.org
Motivated by questions originating from the study of a class of shallow student-teacher
neural networks, methods are developed for the analysis of spurious minima in classes of …
neural networks, methods are developed for the analysis of spurious minima in classes of …
Hidden minima in two-layer relu networks
Y Arjevani - arXiv preprint arXiv:2312.16819, 2023 - arxiv.org
The optimization problem associated to fitting two-layer ReLU networks having $ d $~
inputs, $ k $~ neurons, and labels generated by a target network, is considered. Two …
inputs, $ k $~ neurons, and labels generated by a target network, is considered. Two …
Symmetry & Critical Points
Y Arjevani - arXiv preprint arXiv:2408.14445, 2024 - arxiv.org
Critical points of an invariant function may or may not be symmetric. We prove, however, that
if a symmetric critical point exists, those adjacent to it are generically symmetry breaking …
if a symmetric critical point exists, those adjacent to it are generically symmetry breaking …
On the principle of least symmetry breaking in shallow relu models
Y Arjevani, M Field - arXiv preprint arXiv:1912.11939, 2019 - arxiv.org
We consider the optimization problem associated with fitting two-layer ReLU networks with
respect to the squared loss, where labels are assumed to be generated by a target network …
respect to the squared loss, where labels are assumed to be generated by a target network …