Interactive natural language processing

Z Wang, G Zhang, K Yang, N Shi, W Zhou… - arXiv preprint arXiv …, 2023 - arxiv.org
Interactive Natural Language Processing (iNLP) has emerged as a novel paradigm within
the field of NLP, aimed at addressing limitations in existing frameworks while aligning with …

Embodied AI in education: A review on the body, environment, and mind

B Memarian, T Doleck - Education and Information Technologies, 2024 - Springer
A key feature of embodied education is the participation of the learners' body and mind with
the environment. Yet, little work has been done to review the state of embodied education …

Ical: Continual learning of multimodal agents by transforming trajectories into actionable insights

G Sarch, L Jang, MJ Tarr, WW Cohen, K Marino… - arXiv preprint arXiv …, 2024 - arxiv.org
Large-scale generative language and vision-language models (LLMs and VLMs) excel in
few-shot in-context learning for decision making and instruction following. However, they …

Multitask Multimodal Prompted Training for Interactive Embodied Task Completion

G Pantazopoulos, M Nikandrou, A Parekh… - arXiv preprint arXiv …, 2023 - arxiv.org
Interactive and embodied tasks pose at least two fundamental challenges to existing Vision
& Language (VL) models, including 1) grounding language in trajectories of actions and …

Planning as In-Painting: A Diffusion-Based Embodied Task Planning Framework for Environments under Uncertainty

CF Yang, H Xu, TL Wu, X Gao, KW Chang… - arXiv preprint arXiv …, 2023 - arxiv.org
Task planning for embodied AI has been one of the most challenging problems where the
community does not meet a consensus in terms of formulation. In this paper, we aim to tackle …

HELPER-X: A Unified Instructable Embodied Agent to Tackle Four Interactive Vision-Language Domains with Memory-Augmented Language Models

G Sarch, S Somani, R Kapoor, MJ Tarr… - arXiv preprint arXiv …, 2024 - arxiv.org
Recent research on instructable agents has used memory-augmented Large Language
Models (LLMs) as task planners, a technique that retrieves language-program examples …

[HTML][HTML] The Future of Intelligent Healthcare: A Systematic Analysis and Discussion on the Integration and Impact of Robots Using Large Language Models for …

S Pashangpour, G Nejat - Robotics, 2024 - mdpi.com
The potential use of large language models (LLMs) in healthcare robotics can help address
the significant demand put on healthcare systems around the world with respect to an aging …

Alexa, play with robot: Introducing the first alexa prize simbot challenge on embodied ai

H Shi, L Ball, G Thattai, D Zhang, L Hu, Q Gao… - arXiv preprint arXiv …, 2023 - arxiv.org
The Alexa Prize program has empowered numerous university students to explore,
experiment, and showcase their talents in building conversational agents through …

[PDF][PDF] Program synthesis for robot learning from demonstrations

N Patton, K Rahmani - arXivorg, 2023 - par.nsf.gov
This paper presents a new synthesis-based approach for solving the Learning from
Demonstration (LfD) problem in robotics. Given a set of user demonstrations, the goal of …

Repairs in a Block World: A New Benchmark for Handling User Corrections with Multi-Modal Language Models

J Chiyah-Garcia, A Suglia, A Eshghi - arXiv preprint arXiv:2409.14247, 2024 - arxiv.org
In dialogue, the addressee may initially misunderstand the speaker and respond
erroneously, often prompting the speaker to correct the misunderstanding in the next turn …