Instruction clarification requests in multimodal collaborative dialogue games: Tasks, and an analysis of the CoDraw dataset

B Madureira, D Schlangen - arXiv preprint arXiv:2302.14406, 2023 - arxiv.org
In visual instruction-following dialogue games, players can engage in repair mechanisms in
face of an ambiguous or underspecified instruction that cannot be fully mapped to actions in …

Dialog Acts for Task-Driven Embodied Agents

S Gella, A Padmakumar, P Lange… - arXiv preprint arXiv …, 2022 - arxiv.org
Embodied agents need to be able to interact in natural language understanding task
descriptions and asking appropriate follow up questions to obtain necessary information to …

Do As I Demand, Not As I Say: A Dataset for Developing a Reflective Life-Support Robot

S Tanaka, K Yamasaki, A Yuguchi, S Kawano… - IEEE …, 2024 - ieeexplore.ieee.org
Interactive robots that cooperate with humans must take appropriate actions in response to
their requests. Unfortunately, such requests often have information gaps with their actual …

" Are you telling me to put glasses on the dog?''Content-Grounded Annotation of Instruction Clarification Requests in the CoDraw Dataset

B Madureira, D Schlangen - arXiv preprint arXiv:2306.02377, 2023 - arxiv.org
Instruction Clarification Requests are a mechanism to solve communication problems, which
is very functional in instruction-following interactions. Recent work has argued that the …

Decision-theoretic question generation for situated reference resolution: An empirical study and computational model

F Gervits, G Briggs, A Roque, GA Kadomatsu… - Proceedings of the …, 2021 - dl.acm.org
Dialogue agents that interact with humans in situated environments need to manage
referential ambiguity across multiple modalities and ask for help as needed. However, it is …

Asking the Right Question at the Right Time: Human and Model Uncertainty Guidance to Ask Clarification Questions

A Testoni, R Fernández - arXiv preprint arXiv:2402.06509, 2024 - arxiv.org
Clarification questions are an essential dialogue tool to signal misunderstanding,
ambiguities, and under-specification in language use. While humans are able to resolve …

Taking Action Towards Graceful Interaction: The Effects of Performing Actions on Modelling Policies for Instruction Clarification Requests

B Madureira, D Schlangen - arXiv preprint arXiv:2401.17039, 2024 - arxiv.org
Clarification requests are a mechanism to help solve communication problems, eg due to
ambiguity or underspecification, in instruction-following interactions. Despite their …

Simulating User Agents for Embodied Conversational-AI

D Philipov, V Dongre, G Tur, D Hakkani-Tür - arXiv preprint arXiv …, 2024 - arxiv.org
Embodied agents designed to assist users with tasks must engage in natural language
interactions, interpret instructions, execute actions, and communicate effectively to resolve …

A system for robot concept learning through situated dialogue

B Kane, F Gervits, M Scheutz… - Proceedings of the 23rd …, 2022 - aclanthology.org
Robots operating in unexplored environments with human teammates will need to learn
unknown concepts on the fly. To this end, we demonstrate a novel system that combines a …

Interaction is all You Need? A Study of Robots Ability to Understand and Execute

K Koshti, N Bhavsar - arXiv preprint arXiv:2311.07150, 2023 - arxiv.org
This paper aims to address a critical challenge in robotics, which is enabling them to operate
seamlessly in human environments through natural language interactions. Our primary …