Comparing Distributions by Measuring Differences that Affect Decision Making

Shengjia Zhao · Abhishek Sinha · Yutong He · Aidan Perreault · Jiaming Song · Stefano Ermon

Keywords: [ generative model ]

award Outstanding Paper
[ Abstract ]
[ Visit Poster at Spot I1 in Virtual World ] [ OpenReview
Wed 27 Apr 10:30 a.m. PDT — 12:30 p.m. PDT
Oral presentation: Oral 4: Probablistic Models, Vision
Thu 28 Apr 1 a.m. PDT — 2:30 a.m. PDT


Measuring the discrepancy between two probability distributions is a fundamental problem in machine learning and statistics. We propose a new class of discrepancies based on the optimal loss for a decision task -- two distributions are different if the optimal decision loss is higher on their mixture than on each individual distribution. By suitably choosing the decision task, this generalizes the Jensen-Shannon divergence and the maximum mean discrepancy family. We apply our approach to two-sample tests, and on various benchmarks, we achieve superior test power compared to competing methods. In addition, a modeler can directly specify their preferences when comparing distributions through the decision loss. We apply this property to understanding the effects of climate change on different social and economic activities, evaluating sample quality, and selecting features targeting different decision tasks.

Chat is not available.