Skip to yearly menu bar Skip to main content


A Generalist Agent

Jackie Kay · Sergio Gómez Colmenarejo · Mahyar Bordbar · Mai Giménez · Oriol Vinyals · Scott Reed · Tom Eccles · Ali Razavi · Yury Sulsky · Ashley Edwards · Raia Hadsell · Nando Freitas · Alexander Novikov · Yutian Chen · Jake Bruce · Emilio Parisotto · Konrad Zolna · Jost Springenberg · Nicolas Heess · Gabriel Barth-maron

Halle B #275
[ ] [ Project Page ]
Tue 7 May 1:45 a.m. PDT — 3:45 a.m. PDT


Inspired by progress in large-scale language modeling, we apply a similar approach towards building a single generalist agent beyond the realm of text outputs. The agent, which we refer to as Gato, works as a multi-modal, multi-task, multi-embodiment generalist policy. The same network with the same weights can play Atari, caption images, chat, stack blocks with a real robot arm and much more, deciding based on its context whether to output text, joint torques, button presses, or other tokens. In this report we describe the model and the data, and document the current capabilities of Gato.

Chat is not available.