Robots that use language

S Tellex, N Gopalan, H Kress-Gazit… - Annual Review of …, 2020 - annualreviews.org
This article surveys the use of natural language in robotics from a robotics point of view. To
use human language, robots must map words to aspects of the physical world, mediated by …

A survey on integration of large language models with intelligent robots

Y Kim, D Kim, J Choi, J Park, N Oh, D Park - Intelligent Service Robotics, 2024 - Springer
In recent years, the integration of large language models (LLMs) has revolutionized the field
of robotics, enabling robots to communicate, understand, and reason with human-like …

Translating natural language to planning goals with large-language models

Y Xie, C Yu, T Zhu, J Bai, Z Gong, H Soh - arXiv preprint arXiv:2302.05128, 2023 - arxiv.org
Recent large language models (LLMs) have demonstrated remarkable performance on a
variety of natural language processing (NLP) tasks, leading to intense excitement about their …

Autotamp: Autoregressive task and motion planning with llms as translators and checkers

Y Chen, J Arkin, C Dawson, Y Zhang… - … on robotics and …, 2024 - ieeexplore.ieee.org
For effective human-robot interaction, robots need to understand, plan, and execute
complex, long-horizon tasks described by natural language. Recent advances in large …

Episodic transformer for vision-and-language navigation

A Pashevich, C Schmid, C Sun - Proceedings of the IEEE …, 2021 - openaccess.thecvf.com
Interaction and navigation defined by natural language instructions in dynamic
environments pose significant challenges for neural agents. This paper focuses on …

Language-conditioned imitation learning for robot manipulation tasks

S Stepputtis, J Campbell, M Phielipp… - Advances in …, 2020 - proceedings.neurips.cc
Imitation learning is a popular approach for teaching motor skills to robots. However, most
approaches focus on extracting policy parameters from execution traces alone (ie, motion …

A persistent spatial semantic representation for high-level natural language instruction execution

V Blukis, C Paxton, D Fox, A Garg… - Conference on Robot …, 2022 - proceedings.mlr.press
Natural language provides an accessible and expressive interface to specify long-term tasks
for robotic agents. However, non-experts are likely to specify such tasks with high-level …

nl2spec: Interactively Translating Unstructured Natural Language to Temporal Logics with Large Language Models

M Cosler, C Hahn, D Mendoza, F Schmitt… - … Conference on Computer …, 2023 - Springer
A rigorous formalization of desired system requirements is indispensable when performing
any verification task. This often limits the application of verification techniques, as writing …

Plug in the safety chip: Enforcing constraints for llm-driven robot agents

Z Yang, SS Raman, A Shah… - 2024 IEEE International …, 2024 - ieeexplore.ieee.org
Recent advancements in large language models (LLMs) have enabled a new research
domain, LLM agents, for solving robotics and planning tasks by leveraging the world …

Nl2tl: Transforming natural languages to temporal logics using large language models

Y Chen, R Gandhi, Y Zhang, C Fan - arXiv preprint arXiv:2305.07766, 2023 - arxiv.org
Temporal Logic (TL) can be used to rigorously specify complex high-level specification for
systems in many engineering applications. The translation between natural language (NL) …