In-Person Poster presentation / poster accept
Hierarchical Abstraction for Combinatorial Generalization in Object Rearrangement
Michael Chang · Alyssa Li Dayan · Franziska Meier · Thomas L. Griffiths · Sergey Levine · Amy Zhang
MH1-2-3-4 #78
Keywords: [ rearrangement ] [ symmetry ] [ abstraction ] [ compositionality ] [ objects ] [ binding ] [ combinatorial generalization ] [ independence ] [ slots ] [ graph search ] [ hierarchy ] [ Deep Learning and representational learning ]
Object rearrangement is a challenge for embodied agents because solving these tasks requires generalizing across a combinatorially large set of configurations of entities and their locations. Worse, the representations of these entities are unknown and must be inferred from sensory percepts. We present a hierarchical abstraction approach to uncover these underlying entities and achieve combinatorial generalization from unstructured visual inputs. By constructing a factorized transition graph over clusters of entity representations inferred from pixels, we show how to learn a correspondence between intervening on states of entities in the agent's model and acting on objects in the environment. We use this correspondence to develop a method for control that generalizes to different numbers and configurations of objects, which outperforms current offline deep RL methods when evaluated on simulated rearrangement tasks.