In the past few years, significant progress has been made on deep neural networks (DNNs)
in achieving human-level performance on several long-standing tasks. With the broader …

Deep neural-network-based language models (LMs) are increasingly applied to large-scale
protein sequence data to predict protein function. However, being largely black-box models …

Despite a multitude of empirical studies, little consensus exists on whether neural networks
are able to generalise compositionally, a controversy that, in part, stems from a lack of …

The ability of graph neural networks (GNNs) for distinguishing nodes in graphs has been
recently characterized in terms of the Weisfeiler-Lehman (WL) test for checking graph …

Recent research in mechanistic interpretability has attempted to reverse-engineer
Transformer models by carefully inspecting network weights and activations. However, these …

Deep Learning (DL) has achieved tremendous success in many cutting-edge applications.
However, the state-of-the-art DL systems still suffer from quality issues. While some recent …

Retrieval-based language models (R-LM) model the probability of natural language text by
combining a standard language model (LM) with examples retrieved from an external …

What is the computational model behind a Transformer? Where recurrent neural networks
have direct parallels in finite state machines, allowing clear discussion and thought around …

Explainable Artificial Intelligence (XAI) aims to improve the transparency of machine
learning (ML) pipelines. We systematize the increasingly growing (but fragmented) …

A common method to study deep learning systems is to use simplified model
representations—for example, using singular value decomposition to visualize the model's …