Robot learning in the era of foundation models: A survey

X Xiao, J Liu, Z Wang, Y Zhou, Y Qi, Q Cheng… - arXiv preprint arXiv …, 2023 - arxiv.org
The proliferation of Large Language Models (LLMs) has s fueled a shift in robot learning
from automation towards general embodied Artificial Intelligence (AI). Adopting foundation …

Distilling and retrieving generalizable knowledge for robot manipulation via language corrections

L Zha, Y Cui, LH Lin, M Kwon… - … on Robotics and …, 2024 - ieeexplore.ieee.org
Today's robot policies exhibit subpar performance when faced with the challenge of
generalizing to novel environments. Human corrective feedback is a crucial form of …

Prompt a robot to walk with large language models

YJ Wang, B Zhang, J Chen, K Sreenath - arXiv preprint arXiv:2309.09969, 2023 - arxiv.org
Large language models (LLMs) pre-trained on vast internet-scale data have showcased
remarkable capabilities across diverse domains. Recently, there has been escalating …

Robot utility models: General policies for zero-shot deployment in new environments

H Etukuru, N Naka, Z Hu, S Lee, J Mehu… - arXiv preprint arXiv …, 2024 - arxiv.org
Robot models, particularly those trained with large amounts of data, have recently shown a
plethora of real-world manipulation and navigation capabilities. Several independent efforts …

Prompt, plan, perform: Llm-based humanoid control via quantized imitation learning

J Sun, Q Zhang, Y Duan, X Jiang… - … on Robotics and …, 2024 - ieeexplore.ieee.org
In recent years, reinforcement learning and imitation learning have shown great potential for
controlling humanoid robots' motion. However, these methods typically create simulation …

Towards efficient llm grounding for embodied multi-agent collaboration

Y Zhang, S Yang, C Bai, F Wu, X Li, Z Wang… - arXiv preprint arXiv …, 2024 - arxiv.org
Grounding the reasoning ability of large language models (LLMs) for embodied tasks is
challenging due to the complexity of the physical world. Especially, LLM planning for multi …

Closed-loop open-vocabulary mobile manipulation with gpt-4v

P Zhi, Z Zhang, M Han, Z Zhang, Z Li, Z Jiao… - arXiv preprint arXiv …, 2024 - arxiv.org
Autonomous robot navigation and manipulation in open environments require reasoning
and replanning with closed-loop feedback. We present COME-robot, the first closed-loop …

Self-recovery prompting: Promptable general purpose service robot system with foundation models and self-recovery

M Shirasaka, T Matsushima… - … on Robotics and …, 2024 - ieeexplore.ieee.org
A general-purpose service robot (GPSR), which can execute diverse tasks in various
environments, requires a system with high generalizability and adaptability to tasks and …

LoHoRavens: A Long-Horizon Language-Conditioned Benchmark for Robotic Tabletop Manipulation

S Zhang, P Wicke, LK Şenel, L Figueredo… - arXiv preprint arXiv …, 2023 - arxiv.org
The convergence of embodied agents and large language models (LLMs) has brought
significant advancements to embodied instruction following. Particularly, the strong …

Autonomous interactive correction MLLM for robust robotic manipulation

C Xiong, C Shen, X Li, K Zhou, J Liu… - … Annual Conference on …, 2024 - openreview.net
The ability to reflect on and correct failures is crucial for robotic systems to interact stably with
real-life objects. Observing the generalization and reasoning capabilities of Multimodal …