Reset-Free Lifelong Learning with Skill-Space Planning

Kevin Lu · Aditya Grover · Pieter Abbeel · Igor Mordatch

Keywords: [ lifelong ] [ reset-free ] [ reinforcement learning ]

[ Abstract ]
[ Paper ]
Mon 3 May 9 a.m. PDT — 11 a.m. PDT


The objective of \textit{lifelong} reinforcement learning (RL) is to optimize agents which can continuously adapt and interact in changing environments. However, current RL approaches fail drastically when environments are non-stationary and interactions are non-episodic. We propose \textit{Lifelong Skill Planning} (LiSP), an algorithmic framework for lifelong RL based on planning in an abstract space of higher-order skills. We learn the skills in an unsupervised manner using intrinsic rewards and plan over the learned skills using a learned dynamics model. Moreover, our framework permits skill discovery even from offline data, thereby reducing the need for excessive real-world interactions. We demonstrate empirically that LiSP successfully enables long-horizon planning and learns agents that can avoid catastrophic failures even in challenging non-stationary and non-episodic environments derived from gridworld and MuJoCo benchmarks.

Chat is not available.