Reinforcement Learning under a Multi-agent Predictive State Representation Model: Method and Theory

Zhi Zhang · Zhuoran Yang · Han Liu · Pratap Tokekar · Furong Huang

Keywords: [ multi-agent reinforcement learning ]

[ Abstract ]
[ Visit Poster at Spot H2 in Virtual World ] [ OpenReview
Thu 28 Apr 6:30 p.m. PDT — 8:30 p.m. PDT
Spotlight presentation:

Abstract: This paper proposes a new algorithm for learning the optimal policies under a novel multi-agent predictive state representation reinforcement learning model. Compared to the state-of-the-art methods, the most striking feature of our approach is the introduction of a dynamic interaction graph to the model, which allows us to represent each agent's predictive state by considering the behaviors of its ``neighborhood'' agents. Methodologically, we develop an online algorithm that simultaneously learns the predictive state representation and agent policies. Theoretically, we provide an upper bound of the $L_2$-norm of the learned predictive state representation. Empirically, to demonstrate the efficacy of the proposed method, we provide thorough numerical results on both a MAMuJoCo robotic learning experiment and a multi-agent particle learning environment.

Chat is not available.