Recent efforts have incorporated large language models (LLMs) with external resources (eg,
the Internet) or internal control flows (eg, prompt chaining) for tasks requiring grounding or …

Robots operating in human spaces must be able to engage in natural language interaction,
both understanding and executing instructions, and using conversation to resolve ambiguity …

A long-term goal of AI research is to build intelligent agents that can communicate with
humans in natural language, perceive the environment, and perform real-world tasks. Vision …

Large Language Models (LLMs) have demonstrated impressive planning abilities in single-
agent embodied tasks across various domains. However, their capacity for planning and …

In recent years, several machine learning models have been proposed. They are trained
with a language modelling objective on large-scale text-only data. With such pretraining …

Research at the Interaction Lab focuses on human-agent communication using
conversational Natural Language. The ultimate goal is to create systems where humans and …

Collaborative tasks are ubiquitous activities where a form of communication is required in
order to reach a joint goal. Collaborative building is one of such tasks. We wish to develop …

An ideal integration of autonomous agents in a human world implies that they are able to
collaborate on human terms. In particular, theory of mind plays an important role in …

Abstract The Abstraction and Reasoning Corpus (ARC) is a set of procedural tasks that tests
an agent's ability to flexibly solve novel problems. While most ARC tasks are easy for …

We propose a novel task, G4C, to study teacher-student natural language interactions in a
goal-driven and grounded environment. Dungeons and Dragons (D&D), a role-playing …