Know Your Action Set: Learning Action Relations for Reinforcement Learning

Ayush Jain · Norio Kosaka · Kyung-Min Kim · Joseph Lim


Keywords: [ reinforcement learning ] [ relational reasoning ]

[ Abstract ]
[ Visit Poster at Spot A3 in Virtual World ] [ Slides [ OpenReview
Tue 26 Apr 10:30 a.m. PDT — 12:30 p.m. PDT


Intelligent agents can solve tasks in various ways depending on their available set of actions. However, conventional reinforcement learning (RL) assumes a fixed action set. This work asserts that tasks with varying action sets require reasoning of the relations between the available actions. For instance, taking a nail-action in a repair task is meaningful only if a hammer-action is also available. To learn and utilize such action relations, we propose a novel policy architecture consisting of a graph attention network over the available actions. We show that our model makes informed action decisions by correctly attending to other related actions in both value-based and policy-based RL. Consequently, it outperforms non-relational architectures on applications where the action space often varies, such as recommender systems and physical reasoning with tools and skills. Results and code at .

Chat is not available.