Skip to yearly menu bar Skip to main content


Virtual presentation / poster accept

Distributionally Robust Post-hoc Classifiers under Prior Shifts

Jiaheng Wei · Harikrishna Narasimhan · Ehsan Amid · Wen-Sheng Chu · Yang Liu · Abhishek Kumar

Keywords: [ group robustness ] [ post-hoc scaling ] [ Distributional Robustness ] [ spurious correlations ] [ class imbalance ] [ Deep Learning and representational learning ]


Abstract:

The generalization ability of machine learning models degrades significantly when the test distribution shifts away from the training distribution. We investigate the problem of training models that are robust to shifts caused by changes in the distribution of class-priors or group-priors. The presence of skewed training priors can often lead to the models overfitting to spurious features. Unlike existing methods, which optimize for either the worst or the average performance over classes or groups, our work is motivated by the need for finer control over the robustness properties of the model. We present an extremely lightweight post-hoc approach that performs scaling adjustments to predictions from a pre-trained model, with the goal of minimizing a distributionally robust loss around a chosen target distribution. These adjustments are computed by solving a constrained optimization problem on a validation set and applied to the model during test time. Our constrained optimization objective is inspired from a natural notion of robustness to controlled distribution shifts. Our method comes with provable guarantees and empirically makes a strong case for distributional robust post-hoc classifiers. An empirical implementation is available at https://github.com/weijiaheng/Drops.

Chat is not available.