Transformers have revolutionized vision and natural language processing with their ability to
scale with large datasets. But in robotic manipulation, data is both limited and expensive …

In human environments, robots are expected to accomplish a variety of manipulation tasks
given simple natural language instructions. Yet, robotic manipulation is extremely …

General-purpose robots coexisting with humans in their environment must learn to relate
human language to their perceptions and actions to be useful in a range of daily tasks …

A long-standing goal in robotics is to build robots that can perform a wide range of daily
tasks from perceptions obtained with their onboard sensors and specified only via natural …

Recent works have shown that Large Language Models (LLMs) can be applied to ground
natural language to a wide variety of robot skills. However, in practice, learning multi-task …

We aim to endow a robot with the ability to learn manipulation concepts that link natural
language instructions to motor skills. Our goal is to learn a single multi-task policy that takes …

With the emergence of foundation models (FMs) that are trained on large amounts of data at
scale and adaptable to a wide range of downstream applications, AI is experiencing a …

Task-oriented grasping (TOG) refers to the problem of predicting grasps on an object that
enable subsequent manipulation tasks. To model the complex relationships between …

To perform household tasks, assistive robots receive commands in the form of user
language instructions for tool manipulation. The initial stage involves selecting the intended …

We consider the task of grasping a target object based on a natural language command
query. Previous work primarily focused on localizing the object given the query, which …