Robocook: Long-horizon elasto-plastic object manipulation with diverse tools

H Shi, H Xu, S Clarke, Y Li, J Wu - arXiv preprint arXiv:2306.14447, 2023 - arxiv.org
arXiv preprint arXiv:2306.14447, 2023arxiv.org
Humans excel in complex long-horizon soft body manipulation tasks via flexible tool use:
bread baking requires a knife to slice the dough and a rolling pin to flatten it. Often regarded
as a hallmark of human cognition, tool use in autonomous robots remains limited due to
challenges in understanding tool-object interactions. Here we develop an intelligent robotic
system, RoboCook, which perceives, models, and manipulates elasto-plastic objects with
various tools. RoboCook uses point cloud scene representations, models tool-object …
Humans excel in complex long-horizon soft body manipulation tasks via flexible tool use: bread baking requires a knife to slice the dough and a rolling pin to flatten it. Often regarded as a hallmark of human cognition, tool use in autonomous robots remains limited due to challenges in understanding tool-object interactions. Here we develop an intelligent robotic system, RoboCook, which perceives, models, and manipulates elasto-plastic objects with various tools. RoboCook uses point cloud scene representations, models tool-object interactions with Graph Neural Networks (GNNs), and combines tool classification with self-supervised policy learning to devise manipulation plans. We demonstrate that from just 20 minutes of real-world interaction data per tool, a general-purpose robot arm can learn complex long-horizon soft object manipulation tasks, such as making dumplings and alphabet letter cookies. Extensive evaluations show that RoboCook substantially outperforms state-of-the-art approaches, exhibits robustness against severe external disturbances, and demonstrates adaptability to different materials.
arxiv.org
以上显示的是最相近的搜索结果。 查看全部搜索结果