If generalist robots are to operate in truly unstructured environments, they need to be able to
recognize and reason about novel objects and scenarios. Such objects and scenarios might …

Generative pre-trained models have demonstrated remarkable effectiveness in language
and vision domains by learning useful representations. In this paper, we extend the scope of …

This survey presents an overview of methods for learning from video (LfV) in the context of
reinforcement learning (RL) and robotics. We focus on methods capable of scaling to large …

Foundation models (FMs), large deep learning models pre-trained on vast, unlabeled
datasets, exhibit powerful capabilities in understanding complex patterns and generating …

Teaching robots new skills quickly and conveniently is crucial for the broader adoption of
robotic systems. In this work, we address the problem of one-shot imitation from a single …

Learning from observation (LfO) aims to imitate experts by learning from state-only
demonstrations without requiring action labels. Existing adversarial imitation learning …

Abstract Pre-training for Reinforcement Learning (RL) with purely video data is a valuable
yet challenging problem. Although in-the-wild videos are readily available and inhere a vast …

Learning a generalist embodied agent capable of completing multiple tasks poses
challenges, primarily stemming from the scarcity of action-labeled robotic datasets. In …

Multimodal pretraining has emerged as an effective strategy for the trinity of goals of
representation learning in autonomous robots: 1) extracting both local and global task …

Recently, various pre-training methods have been introduced in vision-based
Reinforcement Learning (RL). However, their generalization ability remains unclear due to …