Workshop
A Roadmap to Never-Ending RL
Feryal Behbahani 路 Khimya Khetarpal 路 Louis Kirsch 路 Rose Wang 路 Annie Xie 路 Adam White 路 Doina Precup
Fri 7 May, 6 a.m. PDT
Humans have a remarkable ability to continually learn and adapt to new scenarios over the duration of their lifetime (Smith & Gasser, 2005). This ability is referred to as never ending learning, also known as continual learning or lifelong learning. Never-ending learning is the constant development of increasingly complex behaviors and the process of building complicated skills on top of those already developed (Ring, 1997), while being able to reapply, adapt and generalize its abilities to new situations. A never-ending learner has the following desiderata
1) it learns behaviors and skills while solving its tasks
2) it invents new subtasks that may later serve as stepping stones
3) it learns hierarchically, i.e. skills learned now can be built upon later
4) it learns without ergodic or resetting assumptions on the underlying (PO)MDP
5) it learns without episode boundaries
6) it learns in a single life without leveraging multiple episodes of experience
There are several facets to building AI agents with never-ending learning abilities. Moreover, different fields have a variety of perspectives to achieving this goal. To this end, we identify key themes for our workshop including cognitive sciences, developmental robotics, agency and abstractions, open-ended learning, world modelling and active inference.
Schedule
Fri 6:00 a.m. - 7:00 a.m.
|
Poster Session #1 ( Poster session ) > link | 馃敆 |
Fri 7:00 a.m. - 7:15 a.m.
|
Organizers Opening Remarks
(
Opening remarks
)
>
|
Feryal Behbahani 路 Louis Kirsch 路 Khimya Khetarpal 路 Rose Wang 路 Annie Xie 馃敆 |
Fri 7:15 a.m. - 7:16 a.m.
|
Speaker & Panelist Introduction #1: Danijar Hafner & Eric Eaton
(
Speaker introduction
)
>
|
Feryal Behbahani 馃敆 |
Fri 7:16 a.m. - 7:31 a.m.
|
Invited Talk #1: Danijar Hafner
(
Invited talk
)
>
SlidesLive Video |
Danijar Hafner 馃敆 |
Fri 7:31 a.m. - 8:00 a.m.
|
Panel #1: Danijar Hafner & Eric Eaton
(
Panel discusion
)
>
|
馃敆 |
Fri 8:00 a.m. - 8:15 a.m.
|
Contributed Talk #1: Continuous Coordination As a Realistic Scenario For Lifelong Learning
(
Contributed talk
)
>
SlidesLive Video |
Akilesh Badrinaaraayanan 路 Hadi Nekoei 路 Aaron Courville 路 Sarath Chandar 馃敆 |
Fri 8:15 a.m. - 8:16 a.m.
|
Speaker & Panelist Introduction #2: Anna Harutyunyan & Martha White
(
Speaker introduction
)
>
|
Feryal Behbahani 馃敆 |
Fri 8:16 a.m. - 8:31 a.m.
|
Invited Talk #2: Anna Harutyunyan
(
Invited talk
)
>
SlidesLive Video |
Anna Harutyunyan 馃敆 |
Fri 8:31 a.m. - 9:00 a.m.
|
Panel #2: Anna Harutyunyan & Martha White
(
Panel discusion
)
>
|
馃敆 |
Fri 9:00 a.m. - 9:15 a.m.
|
Contributed Talk #2: Reward and Optimality Empowerments: Information-Theoretic Measures for Task Complexity in Deep Reinforcement Learning
(
Contributed talk
)
>
|
Hiroki Furuta 路 Tatsuya Matsushima 路 Tadashi Kozuno 路 Yutaka Matsuo 路 Sergey Levine 路 Ofir Nachum 路 Shixiang Gu 馃敆 |
Fri 9:15 a.m. - 9:20 a.m.
|
Break
|
馃敆 |
Fri 9:20 a.m. - 10:05 a.m.
|
Roundtable Panel
(
Panel discusion
)
>
|
Adam White 馃敆 |
Fri 10:05 a.m. - 10:06 a.m.
|
Speaker & Panelist Introduction #3: Joel Lehman & Pierre-Yves Oudeyer
(
Speaker introduction
)
>
|
Louis Kirsch 馃敆 |
Fri 10:06 a.m. - 10:21 a.m.
|
Invited Talk #3: Joel Lehman
(
Invited talk
)
>
SlidesLive Video |
Joel Lehman 馃敆 |
Fri 10:21 a.m. - 10:50 a.m.
|
Panel #3: Joel Lehman & Pierre-Yves Oudeyer
(
Panel discusion
)
>
|
馃敆 |
Fri 10:50 a.m. - 11:05 a.m.
|
Contributed Talk #3: RECON: Rapid Exploration for Open-World Navigation with Latent Goal Models
(
Contributed talk
)
>
SlidesLive Video |
Dhruv Shah 路 Benjamin Eysenbach 路 Nicholas Rhinehart 路 Sergey Levine 馃敆 |
Fri 11:05 a.m. - 11:10 a.m.
|
Break
|
馃敆 |
Fri 11:10 a.m. - 11:11 a.m.
|
Speaker & Panelist Introduction #4: Natalia D铆az-Rodr铆guez & Aleksandra Faust
(
Speaker introduction
)
>
|
Annie Xie 馃敆 |
Fri 11:11 a.m. - 11:26 a.m.
|
Invited Talk #4: Natalia D铆az-Rodr铆guez
(
Invited talk
)
>
SlidesLive Video |
Natalia Diaz Rodriguez 馃敆 |
Fri 11:26 a.m. - 11:55 a.m.
|
Panel #4: Natalia D铆az-Rodr铆guez & Aleksandra Faust
(
Panel discussion
)
>
|
馃敆 |
Fri 11:55 a.m. - 11:56 a.m.
|
Speaker & Panelist Introduction #5: Hyo Gweon & Matt Botvinick
(
Speaker introduction
)
>
|
Rose Wang 馃敆 |
Fri 11:56 a.m. - 12:11 p.m.
|
Invited Talk #5: Hyo Gweon
(
Invited talk
)
>
SlidesLive Video |
Hyowon Gweon 馃敆 |
Fri 12:11 p.m. - 12:40 p.m.
|
Panel #5: Hyo Gweon & Matt Botvinick
(
Panel discussion
)
>
|
馃敆 |
Fri 12:40 p.m. - 12:55 p.m.
|
Closing remarks
(
Closing remarks
)
>
|
Feryal Behbahani 路 Louis Kirsch 路 Khimya Khetarpal 路 Annie Xie 路 Rose Wang 馃敆 |
Fri 12:55 p.m. - 1:55 p.m.
|
Poster Session #2 ( Poster Session ) > link | 馃敆 |
-
|
PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse TD Learning
(
Poster
)
>
SlidesLive Video |
Angelos Filos 路 Clare Lyle 路 Yarin Gal 路 Sergey Levine 路 Natasha Jaques 路 Gregory Farquhar 馃敆 |
-
|
Persistent Reinforcement Learning via Subgoal Curricula
(
Poster
)
>
SlidesLive Video |
Archit Sharma 路 Abhishek Gupta 路 Karol Hausman 路 Sergey Levine 路 Chelsea Finn 馃敆 |
-
|
Fast Inference and Transfer of Compositional Task Structure for Few-shot Task Generalization
(
Poster
)
>
SlidesLive Video |
Sungryull Sohn 路 Hyunjae Woo 路 Jongwook Choi 路 Izzeddin Gur 路 Aleksandra Faust 路 Honglak Lee 馃敆 |
-
|
Multi-Task Reinforcement Learning with Context-based Representations
(
Poster
)
>
SlidesLive Video |
Shagun Sodhani 路 Amy Zhang 路 Joelle Pineau 馃敆 |
-
|
On Lottery Tickets and Minimal Task Representations in Deep Reinforcement Learning
(
Poster
)
>
SlidesLive Video |
Marc Vischer 路 Henning Sprekeler 路 Robert Lange 馃敆 |
-
|
CoMPS: Continual Meta Policy Search
(
Poster
)
>
SlidesLive Video |
Glen Berseth 路 Zhiwei Zhang 路 Chelsea Finn 路 Sergey Levine 馃敆 |
-
|
RL for Autonomous Mobile Manipulation with Applications to Room Cleaning
(
Poster
)
>
SlidesLive Video |
Charles Sun 路 Coline Devin 路 Abhishek Gupta 路 Glen Berseth 路 Sergey Levine 馃敆 |
-
|
OptiDICE: Offline Policy Optimization via Stationary Distribution Correction Estimation
(
Poster
)
>
SlidesLive Video |
Jongmin Lee 路 Wonseok Jeon 路 Byung-Jun Lee 路 Joelle Pineau 路 Kee-Eung Kim 馃敆 |
-
|
COMBO: Conservative Offline Model-Based Policy Optimization
(
Poster
)
>
SlidesLive Video |
Tianhe Yu 路 Aviral Kumar 路 Aravind Rajeswaran 路 Rafael Rafailov 路 Sergey Levine 路 Chelsea Finn 馃敆 |
-
|
Towards Reinforcement Learning in the Continuing Setting
(
Poster
)
>
SlidesLive Video |
Abhishek Naik 路 Zaheer Abbas 路 Adam White 路 Richard Sutton 馃敆 |
-
|
Self-Constructing Neural Networks through Random Mutation
(
Poster
)
>
SlidesLive Video |
Samuel Schmidgall 馃敆 |
-
|
Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Intervention
(
Poster
)
>
SlidesLive Video |
Abhishek Gupta 路 Justin Yu 路 Vikash Kumar 路 Tony Zhao 路 Kelvin Xu 路 Aaron Rovinsky 路 Thomas Devlin 路 Sergey Levine 馃敆 |
-
|
What is Going on Inside Recurrent Meta Reinforcement Learning Agents?
(
Poster
)
>
SlidesLive Video |
Safa Alver 路 Doina Precup 馃敆 |