Optimizing neural networks via koopman operator theory
Koopman operator theory, a powerful framework for discovering the underlying dynamics of
nonlinear dynamical systems, was recently shown to be intimately connected with neural …
nonlinear dynamical systems, was recently shown to be intimately connected with neural …
Renormalization group as a Koopman operator
WT Redman - Physical Review E, 2020 - APS
Koopman operator theory is shown to be directly related to the renormalization group. This
observation allows us, with no assumption of translational invariance, to compute the critical …
observation allows us, with no assumption of translational invariance, to compute the critical …
Iterative Magnitude Pruning as a Renormalisation Group: A Study in The Context of The Lottery Ticket Hypothesis
AA Hassan - arXiv preprint arXiv:2308.03128, 2023 - arxiv.org
This thesis delves into the intricate world of Deep Neural Networks (DNNs), focusing on the
exciting concept of the Lottery Ticket Hypothesis (LTH). The LTH posits that within extensive …
exciting concept of the Lottery Ticket Hypothesis (LTH). The LTH posits that within extensive …
Enhancing Neural Network Differential Equation Solvers
MJH Wright - arXiv preprint arXiv:2301.13146, 2022 - arxiv.org
We motivate the use of neural networks for the construction of numerical solutions to
differential equations. We prove that there exists a feed-forward neural network that can …
differential equations. We prove that there exists a feed-forward neural network that can …
[PDF][PDF] Explaining Transformers through Dynamical Systems Theory
N Okorokov - 2024 - mediatum.ub.tum.de
This thesis explores the operations of Transformer blocks through the lens of Koopman
Operator theory, aiming to provide a deeper understanding of their inner dynamics …
Operator theory, aiming to provide a deeper understanding of their inner dynamics …
Analyzing the Short-Term Dependency in Ultra-High Magnetic Response Systems-Modeling Sequential Data with Non-Recurrent Neural Networks
J Sun, L Li - Procedia Computer Science, 2021 - Elsevier
Recurrent neural network (RNN) is a popular modeling choice for sequential data. However,
empirical experience shows RNNs are often difficult and time-consuming to tune and …
empirical experience shows RNNs are often difficult and time-consuming to tune and …
[图书][B] Mean-Field Approaches for Network Inference and Learning
C Jiang - 2022 - search.proquest.com
Many natural phenomenon such as human activities, wild species evolution and epidemic
spreading, man-made complex systems like the World Wide Web, the power grids and the …
spreading, man-made complex systems like the World Wide Web, the power grids and the …
Efficient Learning of the State in Dynamic Systems From Model-Based to Data-Based
J Sun - 2023 - search.proquest.com
Understanding and predicting states in dynamic systems is a core challenge in many areas
of study. This involves trying to estimate an evolving signal based on past observations or …
of study. This involves trying to estimate an evolving signal based on past observations or …
Error estimation and correction from within neural network differential equation solvers
AS Dogra - arXiv preprint arXiv:2007.04433, 2020 - arxiv.org
Neural Network Differential Equation (NN DE) solvers have surged in popularity due to a
combination of factors: computational advances making their optimization more tractable …
combination of factors: computational advances making their optimization more tractable …
[PDF][PDF] Optimizing Neural Networks via Koopman Operator Theory (Supplemental Material)
As discussed in Sec. 3 of the main text, the computational complexity of Koopman training is
dependent on the approach we take to estimate the action of the Koopman operator on the …
dependent on the approach we take to estimate the action of the Koopman operator on the …