Workshop
A Roadmap to Never-Ending RL
Feryal Behbahani · Khimya Khetarpal · Louis Kirsch · Rose Wang · Annie Xie · Adam White · Doina Precup
Fri 7 May, 6 a.m. PDT
Humans have a remarkable ability to continually learn and adapt to new scenarios over the duration of their lifetime (Smith & Gasser, 2005). This ability is referred to as never ending learning, also known as continual learning or lifelong learning. Never-ending learning is the constant development of increasingly complex behaviors and the process of building complicated skills on top of those already developed (Ring, 1997), while being able to reapply, adapt and generalize its abilities to new situations. A never-ending learner has the following desiderata
1) it learns behaviors and skills while solving its tasks
2) it invents new subtasks that may later serve as stepping stones
3) it learns hierarchically, i.e. skills learned now can be built upon later
4) it learns without ergodic or resetting assumptions on the underlying (PO)MDP
5) it learns without episode boundaries
6) it learns in a single life without leveraging multiple episodes of experience
There are several facets to building AI agents with never-ending learning abilities. Moreover, different fields have a variety of perspectives to achieving this goal. To this end, we identify key themes for our workshop including cognitive sciences, developmental robotics, agency and abstractions, open-ended learning, world modelling and active inference.
Schedule
Fri 6:00 a.m. - 7:00 a.m.
|
Poster Session #1 ( Poster session ) > link | 🔗 |
Fri 7:00 a.m. - 7:15 a.m.
|
Organizers Opening Remarks
(
Opening remarks
)
>
|
Feryal Behbahani · Louis Kirsch · Khimya Khetarpal · Rose Wang · Annie Xie 🔗 |
Fri 7:15 a.m. - 7:16 a.m.
|
Speaker & Panelist Introduction #1: Danijar Hafner & Eric Eaton
(
Speaker introduction
)
>
|
Feryal Behbahani 🔗 |
Fri 7:16 a.m. - 7:31 a.m.
|
Invited Talk #1: Danijar Hafner
(
Invited talk
)
>
SlidesLive Video |
Danijar Hafner 🔗 |
Fri 7:31 a.m. - 8:00 a.m.
|
Panel #1: Danijar Hafner & Eric Eaton
(
Panel discusion
)
>
|
🔗 |
Fri 8:00 a.m. - 8:15 a.m.
|
Contributed Talk #1: Continuous Coordination As a Realistic Scenario For Lifelong Learning
(
Contributed talk
)
>
SlidesLive Video |
Akilesh Badrinaaraayanan · Hadi Nekoei · Aaron Courville · Sarath Chandar 🔗 |
Fri 8:15 a.m. - 8:16 a.m.
|
Speaker & Panelist Introduction #2: Anna Harutyunyan & Martha White
(
Speaker introduction
)
>
|
Feryal Behbahani 🔗 |
Fri 8:16 a.m. - 8:31 a.m.
|
Invited Talk #2: Anna Harutyunyan
(
Invited talk
)
>
SlidesLive Video |
Anna Harutyunyan 🔗 |
Fri 8:31 a.m. - 9:00 a.m.
|
Panel #2: Anna Harutyunyan & Martha White
(
Panel discusion
)
>
|
🔗 |
Fri 9:00 a.m. - 9:15 a.m.
|
Contributed Talk #2: Reward and Optimality Empowerments: Information-Theoretic Measures for Task Complexity in Deep Reinforcement Learning
(
Contributed talk
)
>
|
Hiroki Furuta · Tatsuya Matsushima · Tadashi Kozuno · Yutaka Matsuo · Sergey Levine · Ofir Nachum · Shixiang Gu 🔗 |
Fri 9:15 a.m. - 9:20 a.m.
|
Break
|
🔗 |
Fri 9:20 a.m. - 10:05 a.m.
|
Roundtable Panel
(
Panel discusion
)
>
|
Adam White 🔗 |
Fri 10:05 a.m. - 10:06 a.m.
|
Speaker & Panelist Introduction #3: Joel Lehman & Pierre-Yves Oudeyer
(
Speaker introduction
)
>
|
Louis Kirsch 🔗 |
Fri 10:06 a.m. - 10:21 a.m.
|
Invited Talk #3: Joel Lehman
(
Invited talk
)
>
SlidesLive Video |
Joel Lehman 🔗 |
Fri 10:21 a.m. - 10:50 a.m.
|
Panel #3: Joel Lehman & Pierre-Yves Oudeyer
(
Panel discusion
)
>
|
🔗 |
Fri 10:50 a.m. - 11:05 a.m.
|
Contributed Talk #3: RECON: Rapid Exploration for Open-World Navigation with Latent Goal Models
(
Contributed talk
)
>
SlidesLive Video |
Dhruv Shah · Benjamin Eysenbach · Nicholas Rhinehart · Sergey Levine 🔗 |
Fri 11:05 a.m. - 11:10 a.m.
|
Break
|
🔗 |
Fri 11:10 a.m. - 11:11 a.m.
|
Speaker & Panelist Introduction #4: Natalia Díaz-Rodríguez & Aleksandra Faust
(
Speaker introduction
)
>
|
Annie Xie 🔗 |
Fri 11:11 a.m. - 11:26 a.m.
|
Invited Talk #4: Natalia Díaz-Rodríguez
(
Invited talk
)
>
SlidesLive Video |
Natalia Diaz Rodriguez 🔗 |
Fri 11:26 a.m. - 11:55 a.m.
|
Panel #4: Natalia Díaz-Rodríguez & Aleksandra Faust
(
Panel discussion
)
>
|
🔗 |
Fri 11:55 a.m. - 11:56 a.m.
|
Speaker & Panelist Introduction #5: Hyo Gweon & Matt Botvinick
(
Speaker introduction
)
>
|
Rose Wang 🔗 |
Fri 11:56 a.m. - 12:11 p.m.
|
Invited Talk #5: Hyo Gweon
(
Invited talk
)
>
SlidesLive Video |
Hyowon Gweon 🔗 |
Fri 12:11 p.m. - 12:40 p.m.
|
Panel #5: Hyo Gweon & Matt Botvinick
(
Panel discussion
)
>
|
🔗 |
Fri 12:40 p.m. - 12:55 p.m.
|
Closing remarks
(
Closing remarks
)
>
|
Feryal Behbahani · Louis Kirsch · Khimya Khetarpal · Annie Xie · Rose Wang 🔗 |
Fri 12:55 p.m. - 1:55 p.m.
|
Poster Session #2 ( Poster Session ) > link | 🔗 |
-
|
PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse TD Learning
(
Poster
)
>
SlidesLive Video |
Angelos Filos · Clare Lyle · Yarin Gal · Sergey Levine · Natasha Jaques · Gregory Farquhar 🔗 |
-
|
Persistent Reinforcement Learning via Subgoal Curricula
(
Poster
)
>
SlidesLive Video |
Archit Sharma · Abhishek Gupta · Karol Hausman · Sergey Levine · Chelsea Finn 🔗 |
-
|
Fast Inference and Transfer of Compositional Task Structure for Few-shot Task Generalization
(
Poster
)
>
SlidesLive Video |
Sungryull Sohn · Hyunjae Woo · Jongwook Choi · Izzeddin Gur · Aleksandra Faust · Honglak Lee 🔗 |
-
|
Multi-Task Reinforcement Learning with Context-based Representations
(
Poster
)
>
SlidesLive Video |
Shagun Sodhani · Amy Zhang · Joelle Pineau 🔗 |
-
|
On Lottery Tickets and Minimal Task Representations in Deep Reinforcement Learning
(
Poster
)
>
SlidesLive Video |
Marc Vischer · Henning Sprekeler · Robert Lange 🔗 |
-
|
CoMPS: Continual Meta Policy Search
(
Poster
)
>
SlidesLive Video |
Glen Berseth · Zhiwei Zhang · Chelsea Finn · Sergey Levine 🔗 |
-
|
RL for Autonomous Mobile Manipulation with Applications to Room Cleaning
(
Poster
)
>
SlidesLive Video |
Charles Sun · Coline Devin · Abhishek Gupta · Glen Berseth · Sergey Levine 🔗 |
-
|
OptiDICE: Offline Policy Optimization via Stationary Distribution Correction Estimation
(
Poster
)
>
SlidesLive Video |
Jongmin Lee · Wonseok Jeon · Byung-Jun Lee · Joelle Pineau · Kee-Eung Kim 🔗 |
-
|
COMBO: Conservative Offline Model-Based Policy Optimization
(
Poster
)
>
SlidesLive Video |
Tianhe Yu · Aviral Kumar · Aravind Rajeswaran · Rafael Rafailov · Sergey Levine · Chelsea Finn 🔗 |
-
|
Towards Reinforcement Learning in the Continuing Setting
(
Poster
)
>
SlidesLive Video |
Abhishek Naik · Zaheer Abbas · Adam White · Richard Sutton 🔗 |
-
|
Self-Constructing Neural Networks through Random Mutation
(
Poster
)
>
SlidesLive Video |
Samuel Schmidgall 🔗 |
-
|
Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Intervention
(
Poster
)
>
SlidesLive Video |
Abhishek Gupta · Justin Yu · Vikash Kumar · Tony Zhao · Kelvin Xu · Aaron Rovinsky · Thomas Devlin · Sergey Levine 🔗 |
-
|
What is Going on Inside Recurrent Meta Reinforcement Learning Agents?
(
Poster
)
>
SlidesLive Video |
Safa Alver · Doina Precup 🔗 |