Poster
Reward Constrained Policy Optimization
Chen Tessler · Daniel J Mankowitz · Shie Mannor
Great Hall BC #46
Keywords: [ constrained markov decision process ] [ markov decision process ] [ reinforcement learning ] [ deep learning ]
Solving tasks in Reinforcement Learning is no easy feat. As the goal of the agent is to maximize the accumulated reward, it often learns to exploit loopholes and misspecifications in the reward signal resulting in unwanted behavior. While constraints may solve this issue, there is no closed form solution for general constraints. In this work we present a novel multi-timescale approach for constrained policy optimization, called `Reward Constrained Policy Optimization' (RCPO), which uses an alternative penalty signal to guide the policy towards a constraint satisfying one. We prove the convergence of our approach and provide empirical evidence of its ability to train constraint satisfying policies.
Live content is unavailable. Log in and register to view live content