LLMs with visual inputs, ie, Vision Language Models (VLMs), have the capacity to process
state information as visual-textual prompts and respond with policy decisions in text. We …

Abstract Vision Transformers (ViTs) have become ubiquitous in computer vision. Despite
their success, ViTs lack inductive biases, which can make it difficult to train them with limited …

We explore the problem of computationally generating special images that produce multi-
arrangement optical illusions when physically arranged and viewed in a certain way, which …

Image-generation diffusion models have been fine-tuned to unlock new capabilities such as
image-editing and novel view synthesis. Can we similarly unlock image-generation models …

Diffusion models have demonstrated their capabilities in modeling trajectories of multi-tasks.
However, existing multi-task planners or policies typically rely on task-specific …

Diffusion models (DMs) have emerged as a promising approach for behavior cloning (BC).
Diffusion policies (DP) based on DMs have elevated BC performance to new heights …

This work introduces the Multimodal Diffusion Transformer (MDT), a novel diffusion policy
framework, that excels at learning versatile behavior from multimodal goal specifications …

Diffusion policies have demonstrated robust performance in generative modeling, prompting
their application in robotic manipulation controlled via language descriptions. In this paper …

A common failure mode for policies trained with imitation is compounding execution errors at
test time. When the learned policy encounters states that were not present in the expert …

Visual perception tasks are predominantly solved by Vision Transformer (ViT) architectures,
which, despite their effectiveness, encounter a computational bottleneck due to the quadratic …