SHAPE: SCHEDULE HESSIAN ADAPTIVE PARAMETER ESTIMATION FOR SMOOTHER DIFFUSION OPTIMIZATION
Ritika Lamba
Abstract
Noise schedules control information destruction in diffusion models, yet practice relies on hand-crafted designs (Linear, Cosine) or fixed analytic forms. We introduce SHAPE, a Bayesian optimization framework discovering schedules by minimizing validation loss on 2M-parameter proxy models. On CIFAR-10, our learned schedule achieves a 57\% relative FID improvement over Linear baselines (35.50 vs 82.50) using 50M-parameter U-Nets. Through Hessian analysis, we demonstrate that these gains stem from superior geometric conditioning: SHAPE achieves a spectral anisotropy proxy of $\kappa_{\text{sap}}=3.12$ versus $\kappa_{\text{sap}}=79.44$ for Linear schedules, a 25-fold reduction. We provide two explanations for this result: (1) SNR Uniformity: optimal schedules spontaneously maintain near-linear log-SNR ($R^2=0.987$), rediscovering prior information-theoretic principles through pure optimization; (2) Hessian Conditioning: schedules act as implicit preconditioners that smooth the loss landscape. While absolute performance remains below state-of-the-art methods employing 5--15$\times$ larger models, our work provides evidence that noise schedule design is fundamentally a problem of geometric conditioning rather than signal processing intuition. We validate that schedule quality rankings transfer reliably across scales (Spearman $\rho=0.90$), enabling efficient proxy-based optimization with only 15.7\% computational overhead. We conclude by discussing the adaptation of our spectral measures for indefinite Hessians and the potential for combining SHAPE with modern loss-weighting baselines.
Chat is not available.
Successful Page Load