Application, Methodology, and Theory for Gaussian Processes
Abstract
Gaussian processes are a powerful and flexible class of nonparametric models that use covariance functions, or kernels, to describe correlations across data. In addition to expressing realistic assumptions, correlation between samples acts as a substitute for larger sample sizes to improve predictions. This is demonstrated with an application to remote sensing, in which key components of airborne spectroscopy measurements are correlated to achieve greater accuracy and realism in predictions of atmospheric quantities.
In applying or developing methodology for GP's, scalability is a primary concern because the manipulation of the covariance matrix incurs a cubic complexity in the sample size. This is addressed for the case of GP inference with exponential family observations by the Vecchia-Laplace approximation method. By imposing sparsity in the posterior precision and a second order approximation to the exponential family likelihood, we achieve tractable inference with linear complexity in the sample size.
Using approximations for scalability raises theoretical questions about the tradeoff between efficiency and accuracy as studied in minimax theory, so it is of interest to know what level of approximation can be applied and still preserve the optimality of an estimator. Our work on truncated kernel ridge regression provides an answer for the case of a supremum norm loss and a finite eigenbasis representation of the kernel function. The result matches similar findings in the literature in which the effective dimension of the estimator determines the minimum level of approximation.
Aside from the use of approximation to improve scalability, a nonstationary field can be approximated with a stationary GP. We define and study the spaces that result from taking linear combinations of stationary Hilbert spaces, taking a step towards understanding nonstationary functions and the efficiency of corresponding estimators.
Citation
Zilber, Daniel S (2021). Application, Methodology, and Theory for Gaussian Processes. Doctoral dissertation, Texas A&M University. Available electronically from https : / /hdl .handle .net /1969 .1 /195220.