Robots that use language

S Tellex, N Gopalan, H Kress-Gazit… - Annual Review of …, 2020 - annualreviews.org
This article surveys the use of natural language in robotics from a robotics point of view. To
use human language, robots must map words to aspects of the physical world, mediated by …

[HTML][HTML] A review of verbal and non-verbal human–robot interactive communication

N Mavridis - Robotics and Autonomous Systems, 2015 - Elsevier
In this paper, an overview of human–robot interactive communication is presented, covering
verbal as well as non-verbal aspects. Following a historical introduction, and motivation …

Code as policies: Language model programs for embodied control

J Liang, W Huang, F Xia, P Xu… - … on Robotics and …, 2023 - ieeexplore.ieee.org
Large language models (LLMs) trained on code-completion have been shown to be capable
of synthesizing simple Python programs from docstrings [1]. We find that these code-writing …

Autotamp: Autoregressive task and motion planning with llms as translators and checkers

Y Chen, J Arkin, C Dawson, Y Zhang… - … on robotics and …, 2024 - ieeexplore.ieee.org
For effective human-robot interaction, robots need to understand, plan, and execute
complex, long-horizon tasks described by natural language. Recent advances in large …

Understanding natural language commands for robotic navigation and mobile manipulation

S Tellex, T Kollar, S Dickerson, M Walter… - Proceedings of the …, 2011 - ojs.aaai.org
This paper describes a new model for understanding natural language commands given to
autonomous systems that perform navigation and mobile manipulation in semi-structured …

Weakly supervised learning of semantic parsers for mapping instructions to actions

Y Artzi, L Zettlemoyer - … of the association for computational linguistics, 2013 - direct.mit.edu
The context in which language is used provides a strong signal for learning to recover its
meaning. In this paper, we show it can be used within a grounded CCG semantic parsing …

Learning to parse natural language commands to a robot control system

C Matuszek, E Herbst, L Zettlemoyer, D Fox - Experimental robotics: the …, 2013 - Springer
As robots become more ubiquitous and capable of performing complex tasks, the
importance of enabling untrained users to interact with them has increased. In response …

Toward understanding natural language directions

T Kollar, S Tellex, D Roy, N Roy - 2010 5th ACM/IEEE …, 2010 - ieeexplore.ieee.org
Speaking using unconstrained natural language is an intuitive and flexible way for humans
to interact with robots. Understanding this kind of linguistic input is challenging because …

Ltl2action: Generalizing ltl instructions for multi-task rl

P Vaezipoor, AC Li, RAT Icarte… - … on Machine Learning, 2021 - proceedings.mlr.press
We address the problem of teaching a deep reinforcement learning (RL) agent to follow
instructions in multi-task environments. Instructions are expressed in a well-known formal …

[PDF][PDF] Teaching multiple tasks to an RL agent using LTL

R Toro Icarte, TQ Klassen, R Valenzano… - Proceedings of the 17th …, 2018 - ifaamas.org
Reinforcement Learning (RL) algorithms are capable of learning effective behaviours
through trial and error interactions with their environment [40]. The recent combination of …