Skip to yearly menu bar Skip to main content

In-Person Poster presentation / poster accept

Learning Achievement Structure for Structured Exploration in Domains with Sparse Reward

Zihan Zhou · Animesh Garg

MH1-2-3-4 #124

Keywords: [ Reinforcement Learning ] [ deep reinforcement learning ] [ structured exploration ]


We propose Structured Exploration with Achievements (SEA), a multi-stage reinforcement learning algorithm designed for achievement-based environments, a particular type of environment with an internal achievement set. SEA first uses offline data to learn a representation of the known achievements with a determinant loss function, then recovers the dependency graph of the learned achievements with a heuristic algorithm, and finally interacts with the environment online to learn policies that master known achievements and explore new ones with a controller built with the recovered dependency graph. We empirically demonstrate that SEA can recover the achievement structure accurately and improve exploration in hard domains such as Crafter that are procedurally generated with high-dimensional observations like images.

Chat is not available.