How connectivity structure shapes rich and lazy learning in neural circuits
In theoretical neuroscience, recent work leverages deep learning tools to explore how some
network attributes critically influence its learning dynamics. Notably, initial weight …
network attributes critically influence its learning dynamics. Notably, initial weight …
Evolutionary algorithms as an alternative to backpropagation for supervised training of Biophysical Neural Networks and Neural ODEs
Training networks consisting of biophysically accurate neuron models could allow for new
insights into how brain circuits can organize and solve tasks. We begin by analyzing the …
insights into how brain circuits can organize and solve tasks. We begin by analyzing the …
[HTML][HTML] Transition to chaos separates learning regimes and relates to measure of consciousness in recurrent neural networks
Recurrent neural networks exhibit chaotic dynamics when the variance in their connection
strengths exceed a critical value. Recent work indicates connection variance also modulates …
strengths exceed a critical value. Recent work indicates connection variance also modulates …
How Initial Connectivity Shapes Biologically Plausible Learning in Recurrent Neural Networks
W Liu, X Zhang, YH Liu - arXiv preprint arXiv:2410.11164, 2024 - arxiv.org
The impact of initial connectivity on learning has been extensively studied in the context of
backpropagation-based gradient descent, but it remains largely underexplored in …
backpropagation-based gradient descent, but it remains largely underexplored in …
Does online gradient descent (and variants) still work with biased gradient and variance?
A Al-Tawaha, M Jin - 2024 American Control Conference (ACC), 2024 - ieeexplore.ieee.org
Deterministic bias and stochastic unbiased noise in gradients can affect the performance of
online learning algorithms. While existing studies provide bounds for dynamic regret under …
online learning algorithms. While existing studies provide bounds for dynamic regret under …
Manifold Regularization for Memory-Efficient Training of Deep Neural Networks
One of the prevailing trends in the machine-and deep-learning community is to gravitate
towards the use of increasingly larger models in order to keep pushing the state-of-the-art …
towards the use of increasingly larger models in order to keep pushing the state-of-the-art …
Deep learning frameworks for modeling how neural circuits learn
YH Liu - 2024 - search.proquest.com
The brain's prowess in learning and adapting remains an enigma, particularly in its
approach to the'temporal credit assignment'problem. How do neural circuits determine …
approach to the'temporal credit assignment'problem. How do neural circuits determine …