Transformers have achieved success in both language and vision domains. However, it is
prohibitively expensive to scale them to long sequences such as long documents or high …

Recent developments in large-scale machine learning suggest that by scaling up data,
model size and training time properly, one might observe that improvements in pre-training …

Model efficiency is a critical aspect of developing and deploying machine learning models.
Inference time and latency directly affect the user experience, and some applications have …

Abstract Convolutional Neural Networks (CNNs) have dominated computer vision for years,
due to its ability in capturing locality and translation invariance. Recently, many vision …

Scenic is an open-source (https://github. com/google-research/scenic) JAX library with a
focus on transformer-based models for computer vision research and beyond. The goal of …

Schizophrenia is a socially significant mental disorder resulting frequently in severe forms of
disability. Diagnosis, choice of treatment tactics, and rehabilitation in clinical psychiatry are …

Abstract Vision Transformers (ViT) have recently demonstrated success across a myriad of
computer vision tasks. However, their elevated computational demands pose significant …

We propose a new class of linear Transformers called FourierLearner-Transformers (FLTs),
which incorporate a wide range of relative positional encoding mechanisms (RPEs). These …

Neural language models (LMs) such as GPT-2 estimate the probability distribution over the
next word by a softmax over the vocabulary. The softmax layer produces the distribution …

Matchmaking systems are vital for creating fair matches in online multiplayer games, which
directly affects players' satisfactions and game experience. Most of the matchmaking …