Skip to yearly menu bar Skip to main content


Learning to Reject with a Fixed Predictor: Application to Decontextualization

Christopher Mohri · Daniel Andor · Eunsol Choi · Michael Collins · Anqi Mao · Yutao Zhong

Halle B #228
[ ]
Wed 8 May 1:45 a.m. PDT — 3:45 a.m. PDT

Abstract: We study the problem of classification with a reject option for a fixed predictor, crucial to natural language processing. We introduce a new problem formulation for this scenario, and an algorithm minimizing a new surrogate loss function. We provide a complete theoretical analysis of the surrogate loss function with a strong $H$-consistency guarantee. For evaluation, we choose the \textit{decontextualization} task, and provide a manually-labelled dataset of $2\mathord,000$ examples. Our algorithm significantly outperforms the baselines considered, with a $\sim 25$% improvement in coverage when halving the error rate, which is only $\sim 3$% away from the theoretical limit.

Chat is not available.