Adapting Diffusion Policies to Novel Environments via Policy-Steered Optimization

We propose a novel method of adapting generative robot control policies to succeed in unfamiliar environments with novel runtime constraints. We use a model-based planner to optimize for trajectories that obey both the novel environmental constraints and the implicit task constraints learned by the policy model. We achieve this by evaluating a policy-alignment objective that measures the policy model’s success at reconstructing noised trajectories. We demonstrate this approach’s ability to generalize to two novel simulation environments with obstacles not seen during training.