General sum stochastic games with networked information flow

Inspired by naturally emerging networks of reinforcement learning problems in operation sectors such as supply chains, epidemics, and social networks, we present a stochastic game formulation with networked reward and transition dependencies for multi-agent reinforcement learning (MARL). Despite the rapid development of MARL algorithms, the majority of research efforts are motivated by settings where each pair of players is either collaborating or fully competing. However, in many large-scale networked systems, pairs of interacting players experience general sum-type objectives--- players must collaborate for total profit maximization and locally compete for individual profit maximization. In contrast to popular MARL benchmark environments, stochastic games with general sum objectives have received relatively less attention, in part due to the lack of motivating application and toy examples. In this paper, we address the lack of motivating examples by presenting a networked stochastic game framework with pair-wise general sum objectives and relating the framework to operation sector systems such as supply chains. We discuss specific game features that distinguish this type of stochastic game from existing canonical stochastic game models and specifically outline the impact of these features on the adaptation of popular multi-agent learning paradigms such as individual learning and centralized learning decentralized execution. We conclude with a two player supply chain to benchmark existing MARL algorithms and to contextualize the challenges at hand.