Poster Sat, Apr 25, 2026 • 11:15 AM – 1:45 PM PDT Pavilion 4 P4-#5005

MARTI: A Framework for Multi-Agent LLM Systems Reinforced Training and Inference

Kaiyan Zhang ⋅ Kai Tian ⋅ Runze Liu ⋅ Sihang Zeng ⋅ Xuekai Zhu ⋅ Guoli Jia ⋅ Yuchen Fan ⋅ Xingtai Lv ⋅ Yuxin Zuo ⋅ Che Jiang ⋅ Yuru wang ⋅ Jianyu Wang ⋅ Ermo Hua ⋅ Xinwei Long ⋅ Junqi Gao ⋅ Youbang Sun ⋅ Zhiyuan Ma ⋅ Ganqu Cui ⋅ Ning Ding ⋅ Biqing Qi ⋅ Bowen Zhou

Abstract

We present MARTI (Multi-Agent Reinforced Training and Inference), an open-source framework designed to facilitate scalable and efficient learning of multi-agent LLM systems. MARTI supports centralized multi-agent interactions and distributed policy training, with the added capability of multi-turn asynchronous rollouts to enhance training efficiency. The framework includes dynamic workflows for multi-agent interactions, which integrate both rule-based verifiable rewards and LLM-based generative rewards. We validate the effectiveness of MARTI through comprehensive experiments on diverse mathematical tasks, demonstrating that multi-agent LLM-based systems outperform single-agent systems within the same inference budget after convergence. Our contributions lay the foundation for exploring scalable collaborations within LLM-based multi-agent systems and advancing the capabilities of large reasoning models.