Warp-U MCMC, Stochastic Bridge Sampling and Supervised Functional Principle Component Analysis

Abstract

This dissertation focuses on Bayesian sampling, Bayesian evidence estimation and supervised functional principle component analysis (PCA) and the corresponding theoretical, computational and application challenges. In Bayesian statistics, strategies for estimating normalizing constants often require samples from the underlying target distribution, but obtaining these samples can be challenging, especially when the target is multi-modal. Some Markov chain Monte Carlo (MCMC) and adaptive importance sampling (AIS) methods are specifically designed to address this challenge, such as parallel tempering. However, tuning these algorithms can be time-consuming, and it is typically unclear which estimation strategy should be applied once the samples are obtained. We propose a new adaptive MCMC method to sample from multi-modal target densities and simultaneously perform much of the computation needed for our complementary normalizing constant estimator. Our approach adapts the bridge sampling estimation techniques and proposes a new version of the Warp-U bridge sampling estimator. An important aspect of our overall method is that it requires minimal tuning and is simpler to apply than many competing techniques. The ergodicity of our sampling algorithm is established. In functional data analysis, incorporating covariates into functional PCA can substantially improve the representation efficiency of the principal components and predictive performance. However, many existing functional PCA methods do not make use of covariates, and those that do often have high computational cost or make overly simplistic assumptions that are violated in practice. We propose a new framework, called Covariate Dependent Functional Principal Component Analysis (CD-FPCA), in which both the mean and covariance structure depend on covariates. We propose a corresponding estimation algorithm, which makes use of spline basis representations and roughness penalties, and is substantially more computationally efficient than competing approaches of adequate estimation and prediction accuracy. A key aspect of this work is our novel approach for modeling the covariance function and ensuring that it is symmetric positive semi-definite. We demonstrate the advantages of our three methods through simulation studies and astronomical data analysis.