Overview

How can we learn diverse, dexterous manipulation behaviors without expensive, per-task tuning? Our work proposes two key ideas: a pre-grasp enhanced learning pipeline, and a large-scale dexterous task benchmark.

Introducing PGDM

Our Pre-Grasp informed Dexterous Manipulation (PGDM) framework generates diverse behaviors, without any hyper-parameter tuning. At the core of PGDM is a well known robotics construct, pre-grasps (i.e. the hand-pose preparing for object interaction). Simply moving the robot hand to a pre-grasp position before starting optimization can induce efficient exploration strategies for acquiring complex dexterous behaviors.

TCDM: Benchmarking Dexterous Manipulation at Scale

We create a Trajectory Conditioned Dexterous Manipulation benchmark (TCDM) that contains 50 diverse manipulation tasks defined over multiple objects and robot hands. Tasks for TCDM are defined automatically using exemplar object trajectories from diverse sources (animators, human behaviors, etc.), without any per-task engineering or supervision.

Video

PGDM achieves a tracking error of 5.23e-3 and success rate of 74.5%, when run on all of the 50 diverse tasks in TCDM. In addition, our baseline and ablation studies investigate our pre-grasp primitive in detail, and we even validate PGDM on real hardware. For more information check out our result video below:

Paper

@inproceedings{dasari2023pgdm,
            title={Learning Dexterous Manipulation from Exemplar Object Trajectories and Pre-Grasps},
            author={Dasari, Sudeep and Gupta, Abhinav and Kumar, Vikash},
            booktitle={2023 IEEE International Conference on Robotics and Automation (ICRA)},
            pages={3889--3896},
            year={2023},
	    organization={IEEE}
          }
}

Acknowledgements

This work was conducted while Sudeep Dasari was an intern at Meta AI. In addition, we'd like to recognize thoughtful feedback from Shikhar Bahl, Homanga Bharadhwaj, Yufei Ye, and Sam Powers that greatly improved this paper. Website Template.