An Optimization View on Dynamic Routing Between Capsules

Despite the effectiveness of dynamic routing procedure recently proposed in \citep{sabour2017dynamic}, we still lack a standard formalization of the heuristic and its implications. In this paper, we partially formulate the routing strategy proposed in \citep{sabour2017dynamic} as an optimization problem that minimizes a combination of clustering-like loss and a KL regularization term between the current coupling distribution and its last states. We then introduce another simple routing approach, which enjoys few interesting properties. In an unsupervised perceptual grouping task, we show experimentally that our routing algorithm outperforms the dynamic routing method proposed in \citep{sabour2017dynamic}.