Gaussian Stochastic Weight Averaging for Bayesian Low-Rank Adaptation of Large Language Models
Fine-tuned Large Language Models (LLMs) often suffer from overconfidence and poor
calibration, particularly when fine-tuned on small datasets. To address these challenges, we …
calibration, particularly when fine-tuned on small datasets. To address these challenges, we …
Predictive Coding Networks and Inference Learning: Tutorial and Survey
B van Zwol, R Jefferson, EL Broek - arXiv preprint arXiv:2407.04117, 2024 - arxiv.org
Recent years have witnessed a growing call for renewed emphasis on neuroscience-
inspired approaches in artificial intelligence research, under the banner of $\textit {NeuroAI} …
inspired approaches in artificial intelligence research, under the banner of $\textit {NeuroAI} …
Sparse bayesian neural networks: Bridging model and parameter uncertainty through scalable variational inference
Bayesian neural networks (BNNs) have recently regained a significant amount of attention in
the deep learning community due to the development of scalable approximate Bayesian …
the deep learning community due to the development of scalable approximate Bayesian …
The Empirical Impact of Neural Parameter Symmetries, or Lack Thereof
Many algorithms and observed phenomena in deep learning appear to be affected by
parameter symmetries--transformations of neural network parameters that do not change the …
parameter symmetries--transformations of neural network parameters that do not change the …
Gradient-free variational learning with conditional mixture networks
Balancing computational efficiency with robust predictive performance is crucial in
supervised learning, especially for critical applications. Standard deep learning models …
supervised learning, especially for critical applications. Standard deep learning models …
Linearization Turns Neural Operators into Function-Valued Gaussian Processes
E Magnani, M Pförtner, T Weber, P Hennig - arXiv preprint arXiv …, 2024 - arxiv.org
Modeling dynamical systems, eg in climate and engineering sciences, often necessitates
solving partial differential equations. Neural operators are deep neural networks designed to …
solving partial differential equations. Neural operators are deep neural networks designed to …
Thermodynamic Bayesian Inference
A fully Bayesian treatment of complicated predictive models (such as deep neural networks)
would enable rigorous uncertainty quantification and the automation of higher-level tasks …
would enable rigorous uncertainty quantification and the automation of higher-level tasks …
Reparameterization invariance in approximate Bayesian inference
Current approximate posteriors in Bayesian neural networks (BNNs) exhibit a crucial
limitation: they fail to maintain invariance under reparameterization, ie BNNs assign different …
limitation: they fail to maintain invariance under reparameterization, ie BNNs assign different …
Low-Budget Simulation-Based Inference with Bayesian Neural Networks
A Delaunoy, MB Bonardeaux, S Mishra-Sharma… - arXiv preprint arXiv …, 2024 - arxiv.org
Simulation-based inference methods have been shown to be inaccurate in the data-poor
regime, when training simulations are limited or expensive. Under these circumstances, the …
regime, when training simulations are limited or expensive. Under these circumstances, the …
Bayesian sparsification for deep neural networks with Bayesian model reduction
Deep learning's immense capabilities are often constrained by the complexity of its models,
leading to an increasing demand for effective sparsification techniques. Bayesian …
leading to an increasing demand for effective sparsification techniques. Bayesian …