Poster Sat, Apr 25, 2026 • 11:15 AM – 1:45 PM PDT Pavilion 4 P4-#4617

GEM: A Gym for Generalist LLMs

Zichen Liu ⋅ Anya Sims ⋅ Keyu Duan ⋅ Changyu Chen ⋅ Simon Yu ⋅ Xiangxin Zhou ⋅ Haotian Xu ⋅ Shaopan Xiong ⋅ Bo Liu ⋅ Chenmien Tan ⋅ Weixun Wang ⋅ Hao Zhu ⋅ Weiyan Shi ⋅ Diyi Yang ⋅ Michael Qizhe Shieh ⋅ Yee Whye Teh ⋅ Wee Sun Lee ⋅ Min Lin

Abstract

The training paradigm for large language models (LLMs) is moving from static datasets to experience-based learning, where agents acquire skills via interacting with complex environments. To facilitate this transition we introduce GEM (General Experience Maker), an open-source environment simulator designed for the age of LLMs. Analogous to OpenAI-Gym for traditional reinforcement learning (RL), GEM provides a standardized framework for the environment-agent interface, including asynchronous vectorized execution for high throughput, and flexible wrappers for easy extensibility. GEM also features a diverse suite of environments, robust integrated tools, and single-file example scripts demonstrating using GEM with five popular RL training frameworks. Along with this, we also provide a set of baselines across 24 environments using REINFORCE with Return Batch Normalization (ReBN), which---unlike GRPO---is compatible with the full RL setting of dense per-turn rewards and arbitrary discount factors. We further conduct apple-to-apple benchmarking of PPO, GRPO and REINFORCE in both single- and multi-turn settings using GEM to shed light on the algorithmic designs. GEM also functions as a convenient evaluation toolkit besides a training environment. We hope this framework can help accelerate future agentic LLM research.