Skip to yearly menu bar Skip to main content


Workshop

Online variance-reducing optimization

Nicolas Le Roux · Reza Babanezhad Harikandeh · Pierre-Antoine Manzagol

East Meeting Level 8 + 15 #3

Thu 3 May, 4:30 p.m. PDT

We emphasize the importance of variance reduction in stochastic methods and propose a probabilistic interpretation as a way to store information about past gradients. The resulting algorithm is very similar to the momentum method, with the difference that the weight over past gradients depends on the distance moved in parameter space rather than the number of steps.

Live content is unavailable. Log in and register to view live content