Instruction clarification requests in multimodal collaborative dialogue games: Tasks, and an analysis of the CoDraw dataset
B Madureira, D Schlangen - arXiv preprint arXiv:2302.14406, 2023 - arxiv.org
In visual instruction-following dialogue games, players can engage in repair mechanisms in
face of an ambiguous or underspecified instruction that cannot be fully mapped to actions in …
face of an ambiguous or underspecified instruction that cannot be fully mapped to actions in …
Dialog Acts for Task-Driven Embodied Agents
Embodied agents need to be able to interact in natural language understanding task
descriptions and asking appropriate follow up questions to obtain necessary information to …
descriptions and asking appropriate follow up questions to obtain necessary information to …
Do As I Demand, Not As I Say: A Dataset for Developing a Reflective Life-Support Robot
Interactive robots that cooperate with humans must take appropriate actions in response to
their requests. Unfortunately, such requests often have information gaps with their actual …
their requests. Unfortunately, such requests often have information gaps with their actual …
" Are you telling me to put glasses on the dog?''Content-Grounded Annotation of Instruction Clarification Requests in the CoDraw Dataset
B Madureira, D Schlangen - arXiv preprint arXiv:2306.02377, 2023 - arxiv.org
Instruction Clarification Requests are a mechanism to solve communication problems, which
is very functional in instruction-following interactions. Recent work has argued that the …
is very functional in instruction-following interactions. Recent work has argued that the …
Decision-theoretic question generation for situated reference resolution: An empirical study and computational model
Dialogue agents that interact with humans in situated environments need to manage
referential ambiguity across multiple modalities and ask for help as needed. However, it is …
referential ambiguity across multiple modalities and ask for help as needed. However, it is …
Asking the Right Question at the Right Time: Human and Model Uncertainty Guidance to Ask Clarification Questions
A Testoni, R Fernández - arXiv preprint arXiv:2402.06509, 2024 - arxiv.org
Clarification questions are an essential dialogue tool to signal misunderstanding,
ambiguities, and under-specification in language use. While humans are able to resolve …
ambiguities, and under-specification in language use. While humans are able to resolve …
Taking Action Towards Graceful Interaction: The Effects of Performing Actions on Modelling Policies for Instruction Clarification Requests
B Madureira, D Schlangen - arXiv preprint arXiv:2401.17039, 2024 - arxiv.org
Clarification requests are a mechanism to help solve communication problems, eg due to
ambiguity or underspecification, in instruction-following interactions. Despite their …
ambiguity or underspecification, in instruction-following interactions. Despite their …
Simulating User Agents for Embodied Conversational-AI
Embodied agents designed to assist users with tasks must engage in natural language
interactions, interpret instructions, execute actions, and communicate effectively to resolve …
interactions, interpret instructions, execute actions, and communicate effectively to resolve …
A system for robot concept learning through situated dialogue
Robots operating in unexplored environments with human teammates will need to learn
unknown concepts on the fly. To this end, we demonstrate a novel system that combines a …
unknown concepts on the fly. To this end, we demonstrate a novel system that combines a …
Interaction is all You Need? A Study of Robots Ability to Understand and Execute
This paper aims to address a critical challenge in robotics, which is enabling them to operate
seamlessly in human environments through natural language interactions. Our primary …
seamlessly in human environments through natural language interactions. Our primary …