Scene understanding models seek to extract the latent factors underlying visual observations. But only recently have they started to account for what is arguably the most fundamental of these factors: the 3D geometry of the world around us. In this talk, we investigate recent approaches which learn to infer 3D aware representations from images in a self-supervised way. In particular, we discuss how we may leverage 3D representations for unsupervised object discovery. We conclude by considering current questions, including how 3D geometry should be built into the model structure, how uncertainty can be handled, and how we might improve the scalability of these models.