MatchMaker: Automated Robotic Assembly Asset Generation for Policy Learning in Simulation

Abstract

Robotic assembly remains a significant challenge due to complexities in visual perception, functional grasping, contact-rich manipulation, and performing high-precision tasks. Simulation-based learning and sim-to-real transfer have led to recent success in solving assembly tasks in the presence of object pose variation, perception noise, and control error; however, the development of a generalist (i.e., multi-task) agent for a broad range of assembly tasks has been limited by the need to manually curate assembly assets, which greatly constrains the number and diversity of assembly problems that can be used for policy learning. Inspired by recent success of using Generative AI to scale up robot learning, we propose MatchMaker, a pipeline to automatically generate diverse, simulation-compatible assembly asset pairs to facilitate learning assembly skills. Specifically, MatchMaker can 1) take a simulation-incompatible, interpenetrating asset pair as input, and automatically convert it into a simulation-compatible, interpenetration-free pair, 2) take an arbitrary single asset as input , and generate a geometrically-mating asset to create an asset pair, 3) automatically erode contact surfaces from (1) or (2) according to a user-specified clearance parameter to generate realistic parts.

Overview


MatchMaker automatically generates assembly pairs in three stages:
  1. Contact-Surface Detection: Identifies the assembly axis and contact faces of an initially sampled asset for the assembly pair.
  2. Shape Completion: Utilizes diffusion models to complete the rest of the asset, generating the second asset of the pair.
  3. Clearance Specification: Erodes the contacting surfaces to prevent interpenetration and achieve the desired clearance.
We further demonstrate that our generated asset pairs are simulatable by training assembly policies within NVIDIA Isaac Simulator and realizable by deploying the learned policies with 3D-printed assets in the real world.
Teaser Image

Pipeline

Contact-Surface Detection GIF
Shape Completion GIF
Clearance Specification GIF