Skip to yearly menu bar Skip to main content


In-Person Poster presentation / top 5% paper

Image to Sphere: Learning Equivariant Features for Efficient Pose Prediction

David Klee · Ondrej Biza · Robert Platt · Robin Walters

MH1-2-3-4 #62

Keywords: [ symmetry ] [ SO(3) ] [ equivariance ] [ sample efficiency ] [ pose detection ] [ Deep Learning and representational learning ]


Abstract: Predicting the pose of objects from a single image is an important but difficult computer vision problem. Methods that predict a single point estimate do not predict the pose of objects with symmetries well and cannot represent uncertainty. Alternatively, some works predict a distribution over orientations in $\mathrm{SO}(3)$. However, training such models can be computation- and sample-inefficient. Instead, we propose a novel mapping of features from the image domain to the 3D rotation manifold. Our method then leverages $\mathrm{SO}(3)$ equivariant layers, which are more sample efficient, and outputs a distribution over rotations that can be sampled at arbitrary resolution. We demonstrate the effectiveness of our method at object orientation prediction, and achieve state-of-the-art performance on the popular PASCAL3D+ dataset. Moreover, we show that our method can model complex object symmetries, without any modifications to the parameters or loss function. Code is available at \url{https://dmklee.github.io/image2sphere}.

Chat is not available.