Bayesian Context Aggregation for Neural Processes

Michael Volpp · Fabian Fl├╝renbrock · Lukas Grossberger · Christian Daniel · Gerhard Neumann


Keywords: [ multi-task learning ] [ meta learning ] [ neural processes ] [ Aggregation Methods ] [ Latent Variable Models ] [ Deep Sets ]

[ Abstract ]
[ Slides [ Paper ]
Tue 4 May 1 a.m. PDT — 3 a.m. PDT


Formulating scalable probabilistic regression models with reliable uncertainty estimates has been a long-standing challenge in machine learning research. Recently, casting probabilistic regression as a multi-task learning problem in terms of conditional latent variable (CLV) models such as the Neural Process (NP) has shown promising results. In this paper, we focus on context aggregation, a central component of such architectures, which fuses information from multiple context data points. So far, this aggregation operation has been treated separately from the inference of a latent representation of the target function in CLV models. Our key contribution is to combine these steps into one holistic mechanism by phrasing context aggregation as a Bayesian inference problem. The resulting Bayesian Aggregation (BA) mechanism enables principled handling of task ambiguity, which is key for efficiently processing context information. We demonstrate on a range of challenging experiments that BA consistently improves upon the performance of traditional mean aggregation while remaining computationally efficient and fully compatible with existing NP-based models.

Chat is not available.