On the Fisher Geometry of Diffusion Models' Latent Space

In this paper, we advance the understanding of the geometry of the latent space of diffusion models through the lens of information geometry. In particular, we consider the latent variable $\theta=(x_t,t)$ that indexes a family of denoising distributions $p(x_0|x_t)$ across all noise levels, which naturally defines a statistical manifold endowed with the Fisher-Rao metric. We study how the choice of noise schedule affects the geometry of this manifold, and how the geometry evolves along the denoising process. We show that, under mild assumptions, the statistical manifolds induced by different noise schedules are isometric, and we demonstrate a link between the curvature and phase-transition-like behavior.