Energy-Based Models: Current Perspectives, Challenges, and Opportunities

Marc Dymetman, Adji Bousso Dieng, Hady Elsahar, Igor Mordatch, Marc'Aurelio Ranzato


Energy-Based Models (EBMs) are a learning framework that assigns a quality score to any given input, its energy; contrary to
probabilistic models, there is no a priori requirement that these scores be normalized (i.e. sum to one). Energies are typically
computed through a neural network, and training an EBM corresponds to shaping the energy function such that data points nearby the underlying data manifold are associated with lower energies than data points that are far from it. Not imposing normalization affords a great power and flexibility to the modelling process, e.g. in terms of combining energies, on conditioning on certain variables, of computing global scores on complex structured objects, or on expressing prior
knowledge. However, this freedom comes with significant technical challenges, in terms of learning and inference.

A strong comeback of EBMs is currently underway. This ICLR-2021 Workshop is the opportunity to increase awareness about the diversity of works in this area, to discuss current challenges, and to encourage cross-pollination between different communities around this topic.

Chat is not available.

Timezone: »


Fri 5:50 a.m. - 6:00 a.m.

Intro by Organisers

Fri 6:00 a.m. - 7:00 a.m.

The talk by Prof. LeCun will be given live, including Q/A at the end.

Fri 7:00 a.m. - 7:02 a.m.
Intro for Yingzhen (Intro for invited speaker)
Fri 7:02 a.m. - 7:52 a.m.
Invited Talk: EBM inference & learning: A personal story (Invited Talk: 50' prerecorded + 8' Q/A live on Zoom)   
Yingzhen Li
Fri 7:52 a.m. - 8:00 a.m.
Q/A on Yingzhen's talk (Q/A)
Fri 8:00 a.m. - 9:00 a.m.
Panel: Yann LeCun, Yingzhen Li, Will Grathwohl --- Focus: Probabilistic vs. Non-Probabilistic Approaches to EBMs (Discussion Panel)
Fri 9:00 a.m. - 9:25 a.m.

Prerecorded duration 25 mns: Improved Contrastive Divergence Training of Energy Based Models

Yilun Du
Fri 9:25 a.m. - 9:30 a.m.
Q/A for Yilun Du (Q/A)
Fri 9:30 a.m. - 9:55 a.m.

Prerecorded: Duration 25 mns Conjugate Energy-Based Models

Hao Wu
Fri 9:55 a.m. - 10:00 a.m.
Q/A for Hao Wu (Q/A)
Fri 10:00 a.m. - 11:00 a.m.
Break (meet on GatherTown if you wish: (Break)
Fri 11:00 a.m. - 12:30 p.m.

All accepted papers (including oral) have a spot on GatherTown where they can share their screen and discuss their work. No actual "posters" are required, but showing slides can be useful to support explanations.

For links to the pdfs of papers, see the Workshop Website:

Fri 12:30 p.m. - 12:32 p.m.
Intro for invited speaker Ermon (Intro for invited speaker)
Fri 12:32 p.m. - 1:00 p.m.
Invited Talk Live: Generative Modeling by Estimating Gradients of the Data Distribution (Live Q/A at the end) (Invited Talk Live)
Stefano Ermon
Fri 1:00 p.m. - 1:02 p.m.
Intro for invited speaker Marks (Intro for invited speaker)
Fri 1:02 p.m. - 2:00 p.m.
Invited Talk Live: Can EBMs help solve important biological challenges? (Live Q/A at the end) (Invited Talk Live + Q/A at the end)
Debora Marks
Fri 2:00 p.m. - 2:02 p.m.
Intro for invited speaker Scellier (Intro for invited speaker)
Fri 2:02 p.m. - 2:52 p.m.
Invited Talk: A deep learning theory for neural networks grounded in physics (Prerecorded Invited Talk)   
Benjamin Scellier
Fri 2:52 p.m. - 3:00 p.m.
Q/A on Scellier's talk (Q/A live)
Fri 3:00 p.m. - 3:25 p.m.

Prerecorded Duration 25 mns Contributed Talk: Graph EBM for Molecular Graph Generation

Ryuichiro Hataya
Fri 3:25 p.m. - 3:30 p.m.
Q/A for Ryuichiro Hataya (Q/A)
Fri 3:30 p.m. - 3:55 p.m.

Prerecorded duration 25 mns Contributed Talk: Energy-Based Models for Continual Learning

Shuang Li
Fri 3:55 p.m. - 4:00 p.m.
Q/A for Shuang Li (Q/A)
Fri 4:00 p.m. - 4:10 p.m.
Concluding remarks

Prerecorded video --- not livestreamed during the workshop, but available to ICLR attendees

Duration: 5 mins

Meng Liu