Maniskill2: A unified benchmark for generalizable manipulation skills

J Gu, F Xiang, X Li, Z Ling, X Liu, T Mu, Y Tang… - arXiv preprint arXiv …, 2023 - arxiv.org
Generalizable manipulation skills, which can be composed to tackle long-horizon and
complex daily chores, are one of the cornerstones of Embodied AI. However, existing …

Dynamic visual reasoning by learning differentiable physics models from video and language

M Ding, Z Chen, T Du, P Luo… - Advances In Neural …, 2021 - proceedings.neurips.cc
In this work, we propose a unified framework, called Visual Reasoning with Differ-entiable
Physics (VRDP), that can jointly learn visual concepts and infer physics models of objects …

Global planning for contact-rich manipulation via local smoothing of quasi-dynamic contact models

T Pang, HJT Suh, L Yang… - IEEE Transactions on …, 2023 - ieeexplore.ieee.org
The empirical success of reinforcement learning (RL) in contact-rich manipulation leaves
much to be understood from a model-based perspective, where the key difficulties are often …

RoboCraft: Learning to see, simulate, and shape elasto-plastic objects in 3D with graph networks

H Shi, H Xu, Z Huang, Y Li… - The International Journal …, 2024 - journals.sagepub.com
Modeling and manipulating elasto-plastic objects are essential capabilities for robots to
perform complex industrial and household interaction tasks (eg, stuffing dumplings, rolling …

Robocook: Long-horizon elasto-plastic object manipulation with diverse tools

H Shi, H Xu, S Clarke, Y Li, J Wu - arXiv preprint arXiv:2306.14447, 2023 - arxiv.org
Humans excel in complex long-horizon soft body manipulation tasks via flexible tool use:
bread baking requires a knife to slice the dough and a rolling pin to flatten it. Often regarded …

Accelerated policy learning with parallel differentiable simulation

J Xu, V Makoviychuk, Y Narang, F Ramos… - arXiv preprint arXiv …, 2022 - arxiv.org
Deep reinforcement learning can generate complex control policies, but requires large
amounts of training data to work effectively. Recent work has attempted to address this issue …

An end-to-end differentiable framework for contact-aware robot design

J Xu, T Chen, L Zlokapa, M Foshey, W Matusik… - arXiv preprint arXiv …, 2021 - arxiv.org
The current dominant paradigm for robotic manipulation involves two separate stages:
manipulator design and control. Because the robot's morphology and how it can be …

Learning neural constitutive laws from motion observations for generalizable pde dynamics

P Ma, PY Chen, B Deng… - International …, 2023 - proceedings.mlr.press
We propose a hybrid neural network (NN) and PDE approach for learning generalizable
PDE dynamics from motion observations. Many NN approaches learn an end-to-end model …

Pac-nerf: Physics augmented continuum neural radiance fields for geometry-agnostic system identification

X Li, YL Qiao, PY Chen, KM Jatavallabhula… - arXiv preprint arXiv …, 2023 - arxiv.org
Existing approaches to system identification (estimating the physical parameters of an
object) from videos assume known object geometries. This precludes their applicability in a …

Learning foresightful dense visual affordance for deformable object manipulation

R Wu, C Ning, H Dong - Proceedings of the IEEE/CVF …, 2023 - openaccess.thecvf.com
Understanding and manipulating deformable objects (eg, ropes and fabrics) is an essential
yet challenging task with broad applications. Difficulties come from complex states and …