Non Parametric Aleatoric Uncertainty Quantification with Neural Networks

Classic methods for aleatoric uncertainty quantification in regression settings make assumptions about the distribution of noise in the dependent variable. Incorrect assumptions can lead to poor model performance and unreliable uncertainty estimates. In this paper, we introduce a simple method for non-parametric aleatoric uncertainty quantification. In particular, we train a neural network model for binary classification. The inputs to our binary classifier are the independent variables and a sample from the marginal distribution of the dependent variable. This binary classifier is trained to predict whether the sample from the marginal distribution of the dependent variable is greater than the dependent variable corresponding to independent variables in the input. Our method can be used for not only quantifying aleatoric uncertainty but also estimating the conditional distribution of the dependent variable.