Model-Based Meta-Learning for Algorithm Discovery

Reinforcement Learning (RL) algorithms are typically hand-crafted through a slow and iterative scientific process. While meta-RL promises to automate algorithm discovery, research into meta-RL has been held back by the large computational requirements of simulating environments for meta-training. In this work, we introduce Model-Based Meta-Learning (MBML), a novel approach that uses learned world models as an efficient alternative to environment simulation. We show that MBML matches the performance of traditional online meta-RL at a fraction of the time and compute, without needing perfect world models. By substantially reducing the computational cost of meta-training, MBML lowers the barrier to entry for meta-RL research and enables algorithm discovery at a larger scale.