While there has been progress in developing non-vacuous generalization bounds for deep
neural networks, these bounds tend to be uninformative about why deep learning works. In …

Graph neural networks (GNNs) have limited expressive power, failing to represent many
graph classes correctly. While more expressive graph representation learning (GRL) …

Despite the empirical successes of self-supervised learning (SSL) methods, it is unclear
what characteristics of their representations lead to high downstream accuracies. In this …

Treating neural network inputs and outputs as random variables, we characterize the
structure of neural networks that can be used to model data that are invariant or equivariant …

Most data is automatically collected and only ever" seen" by algorithms. Yet, data
compressors preserve perceptual fidelity rather than just the information needed by …

Abstract Models trained on one set of domains often suffer performance drops on unseen
domains, eg, when wildlife monitoring models are deployed in new camera locations. In this …

Graph neural networks (GNNs) are commonly described as being permutation equivariant
with respect to node relabeling in the graph. This symmetry of GNNs is often compared to …

Shortcut learning, in which models make use of easy-to-represent but unstable associations,
is a major failure mode for robust machine learning. We study a flexible, causally-motivated …

The explicit incorporation of task-specific inductive biases through symmetry has emerged
as a general design precept in the development of high-performance machine learning …

Abstract Language models pretrained on large collections of tabular data have
demonstrated their effectiveness in several downstream tasks. However, many of these …