Improving Single Noise Level Denoising Samplers with Restricted Gaussian Oracles

Practical generative modeling pipelines and diffusion Monte-Carlo schemes, which adapt diffusion models for sampling from unnormalized log-densities, both rely on denoisers (or score estimates) at different noise scales. This complicates the sampling process as denoising schedules require careful tuning and nested inner-MCMC loops. In this work, we propose a single noise level sampling procedure that only requires a single low-noise denoiser. Our framework results from improvements we bring to the multimeasurement Walk-Jump sampler of Saremi et al. 2021 by mixing in ideas from the proximal sampler of Shen et al. 2020. Our analysis shows that annealing (or multiple noise scales) is unnecessary if one is willing to pay an increased memory cost. We demonstrate this by proposing an \emph{entirely log-concave} sampling framework.