A Coreset-based, Tempered Variational Posterior for Accurate and Scalable Stochastic Gaussian Process Inference

We present a novel stochastic variational Gaussian process (\(\mathcal{GP}\)) inference method, based on a posterior over a learnable set of weighted pseudo input-output points (coresets). Instead of a free-form variational family, the proposed coreset-based, variational tempered family for \(\mathcal{GP}\)s (CVTGP) is defined in terms of the \(\mathcal{GP}\) prior and the data-likelihood; hence, accommodating the modeling inductive biases. We derive CVTGP's lower bound for the log-marginal likelihood via marginalization of the proposed posterior over latent \(\mathcal{GP}\) coreset variables, and show it is amenable to stochastic optimization. CVTGP reduces the learnable parameter size to \(\mathcal{O}(M)\), enjoys numerical stability, and maintains \(\mathcal{O}(M^3)\) time- and \(\mathcal{O}(M^2)\) space-complexity, by leveraging a coreset-based tempered posterior that, in turn, provides sparse and explainable representations of the data. Results on simulated and real-world regression problems with Gaussian observation noise validate that CVTGP provides better evidence lower-bound estimates and predictive root mean squared error than alternative stochastic \(\mathcal{GP}\) inference methods.