Large language models for robotics: A survey

F Zeng, W Gan, Y Wang, N Liu, PS Yu - arXiv preprint arXiv:2311.07226, 2023 - arxiv.org
The human ability to learn, generalize, and control complex manipulation tasks through multi-
modality feedback suggests a unique capability, which we refer to as dexterity intelligence …

A survey on deep learning and deep reinforcement learning in robotics with a tutorial on deep reinforcement learning

EF Morales, R Murrieta-Cid, I Becerra… - Intelligent Service …, 2021 - Springer
This article is about deep learning (DL) and deep reinforcement learning (DRL) works
applied to robotics. Both tools have been shown to be successful in delivering data-driven …

Lm-nav: Robotic navigation with large pre-trained models of language, vision, and action

D Shah, B Osiński, S Levine - Conference on robot …, 2023 - proceedings.mlr.press
Goal-conditioned policies for robotic navigation can be trained on large, unannotated
datasets, providing for good generalization to real-world settings. However, particularly in …

Navigation with large language models: Semantic guesswork as a heuristic for planning

D Shah, MR Equi, B Osiński, F Xia… - … on Robot Learning, 2023 - proceedings.mlr.press
Navigation in unfamiliar environments presents a major challenge for robots: while mapping
and planning techniques can be used to build up a representation of the world, quickly …

Maskvit: Masked visual pre-training for video prediction

A Gupta, S Tian, Y Zhang, J Wu, R Martín-Martín… - arXiv preprint arXiv …, 2022 - arxiv.org
The ability to predict future visual observations conditioned on past observations and motor
commands can enable embodied agents to plan solutions to a variety of tasks in complex …

ViNT: A foundation model for visual navigation

D Shah, A Sridhar, N Dashora, K Stachowicz… - arXiv preprint arXiv …, 2023 - arxiv.org
General-purpose pre-trained models (" foundation models") have enabled practitioners to
produce generalizable solutions for individual machine learning problems with datasets that …

Badgr: An autonomous self-supervised learning-based navigation system

G Kahn, P Abbeel, S Levine - IEEE Robotics and Automation …, 2021 - ieeexplore.ieee.org
Mobile robot navigation is typically regarded as a geometric problem, in which the robot's
objective is to perceive the geometry of the environment in order to plan collision-free paths …

Behavior: Benchmark for everyday household activities in virtual, interactive, and ecological environments

S Srivastava, C Li, M Lingelbach… - … on robot learning, 2022 - proceedings.mlr.press
We introduce BEHAVIOR, a benchmark for embodied AI with 100 activities in simulation,
spanning a range of everyday household chores such as cleaning, maintenance, and food …

Socially compliant navigation dataset (scand): A large-scale dataset of demonstrations for social navigation

H Karnan, A Nair, X Xiao, G Warnell… - IEEE Robotics and …, 2022 - ieeexplore.ieee.org
Social navigation is the capability of an autonomous agent, such as a robot, to navigate in a
“socially compliant” manner in the presence of other intelligent agents such as humans. With …

igibson 1.0: A simulation environment for interactive tasks in large realistic scenes

B Shen, F Xia, C Li, R Martín-Martín… - 2021 IEEE/RSJ …, 2021 - ieeexplore.ieee.org
We present iGibson 1.0, a novel simulation environment to develop robotic solutions for
interactive tasks in large-scale realistic scenes. Our environment contains 15 fully interactive …