New Variational and Sampling Algorithms for Large-scale Bayesian Model Selection Problems

Abstract

Significant progress has been achieved in computer inference for Bayesian models during the past few decades. There has been continuous improvement in a wide variety of computational tools. In this thesis, we extend several Bayesian computation tools.

In the first project, we study the Bayesian multi-task variable selection problem, where the goal is to select activated variables for multiple related data sets simultaneously. Our proposed method generalizes the spike-and-slab prior to multiple data sets, and we prove its posterior consistency in high-dimensional regimes. To compute the posterior distribution, we propose a novel variational Bayes algorithm based on the recently developed "sum of single effects" model. Finally, we apply our method to the joint learning of multiple directed acyclic graphical models. Both simulation studies and real-data analysis are performed to show the effectiveness of the proposed method.

On large-dimensional discrete state spaces, informed Markov chain Monte Carlo (MCMC) algorithms have been proposed as scalable methods for Bayesian posterior calculation. Informed Importance Tempering (IIT), a recently introduced category of Markov chain Monte Carlo techniques, combines importance sampling and informed local proposals. In the second project, we begin by proposing a new MCMC framework generalizing the IIT sampler, which includes the standard Metropolis-Hastings algorithm as a special case and opens the door to devising a new class of MCMC algorithms. Next, we combine IIT with the simulated tempering (ST) method, which includes the standard ST as a special case. A simulation of multimodal variable selection shows that our proposed method can easily deal with multimodality.