## Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models

### Fan Bao · Chongxuan Li · Jun Zhu · Bo Zhang

Keywords: [ diffusion probabilistic models ] [ generative models ]

 Outstanding Paper
[ Abstract ]
Mon 25 Apr 2:30 a.m. PDT — 4:30 a.m. PDT

Oral presentation: Oral 4: Probablistic Models, Vision
Thu 28 Apr 1 a.m. PDT — 2:30 a.m. PDT

Abstract: Diffusion probabilistic models (DPMs) represent a class of powerful generative models. Despite their success, the inference of DPMs is expensive since it generally needs to iterate over thousands of timesteps. A key problem in the inference is to estimate the variance in each timestep of the reverse process. In this work, we present a surprising result that both the optimal reverse variance and the corresponding optimal KL divergence of a DPM have analytic forms w.r.t. its score function. Building upon it, we propose \textit{Analytic-DPM}, a training-free inference framework that estimates the analytic forms of the variance and KL divergence using the Monte Carlo method and a pretrained score-based model. Further, to correct the potential bias caused by the score-based model, we derive both lower and upper bounds of the optimal variance and clip the estimate for a better result. Empirically, our analytic-DPM improves the log-likelihood of various DPMs, produces high-quality samples, and meanwhile enjoys a $20\times$ to $80\times$ speed up.

Chat is not available.