A Roadmap to Never-Ending RL

Humans have a remarkable ability to continually learn and adapt to new scenarios over the duration of their lifetime (Smith & Gasser, 2005). This ability is referred to as never ending learning, also known as continual learning or lifelong learning. Never-ending learning is the constant development of increasingly complex behaviors and the process of building complicated skills on top of those already developed (Ring, 1997), while being able to reapply, adapt and generalize its abilities to new situations. A never-ending learner has the following desiderata

1) it learns behaviors and skills while solving its tasks
2) it invents new subtasks that may later serve as stepping stones
3) it learns hierarchically, i.e. skills learned now can be built upon later
4) it learns without ergodic or resetting assumptions on the underlying (PO)MDP
5) it learns without episode boundaries
6) it learns in a single life without leveraging multiple episodes of experience

There are several facets to building AI agents with never-ending learning abilities. Moreover, different fields have a variety of perspectives to achieving this goal. To this end, we identify key themes for our workshop including cognitive sciences, developmental robotics, agency and abstractions, open-ended learning, world modelling and active inference.