In this paper, we apply the self-attention from the state-of-the-art Transformer in Attention Is
All You Need for the first time to a data-driven operator learning problem related to partial …

As a common mental disorder, depression becomes a major threat to human health and
may even heavily influence one's daily life. Considering this background, it is necessary to …

We propose FMMformers, a class of efficient and flexible transformers inspired by the
celebrated fast multipole method (FMM) for accelerating interacting particle simulation. FMM …

We propose a two-stage memory retrieval dynamics for modern Hopfield models, termed
$\mathtt {U\text {-} Hop} $, with enhanced memory capacity. Our key contribution is a …

Transformers have achieved remarkable success in sequence modeling and beyond but
suffer from quadratic computational and memory complexities with respect to the length of …

Time series forecasting is essential for many practical applications, with the adoption of
transformer-based models on the rise due to their impressive performance in NLP and CV …

We present and review an algorithmic and theoretical framework for improving neural
network architecture design via momentum. As case studies, we consider how momentum …

Linearization of attention using various kernel approximation and kernel learning techniques
has shown promise. Past methods use a subset of combinations of component functions and …

Few-shot learning assumes that we have a very small dataset for each task and trains a
model on the set of tasks. For real-world problems, however, the amount of available data is …

We present a novel bi-directional Transformer architecture (BiXT) for which computational
cost and memory consumption scale linearly with input size, but without suffering the drop in …