Data-Efficient Design and Analysis Methodologies for Computer and Physical Experiments

Abstract

Data science for experimentation, including the rapidly growing area of the design and analysis of computer experiments, aims to use statistical approaches to collect and analyze (physical or virtual) experimental responses and facilitate decision-making. The cost for each run of an experiment can be expensive. This dissertation proposes novel data-efficient methodologies to tackle three different challenges in this field. The first two are regarding computer experiments, and the third one is regarding physical experiments. The first work aims to reconstruct the input-output relationship (surrogate model) given by the computer code via scattered evaluations with small sizes based on Gaussian process regression. Traditional isotropic Gaussian process models suffer from the curse of dimensionality when the input dimension is relatively high given limited data points. Gaussian process models with additive correlation functions are scalable to dimensionality, but they are more restrictive as they only work for additive functions. In the first work, we consider a projection pursuit model in which the nonparametric part is driven by an additive Gaussian process regression. We choose the dimension of the additive function higher than the original input dimension and call this strategy “dimension expansion”. We show that dimension expansion can help approximate more complex functions. A gradient descent algorithm is proposed for model training based on the maximum likelihood estimation. Simulation studies show that the proposed method outperforms the traditional Gaussian process models. The second work focuses on the designs of experiments (DoE) of multi-fidelity computer experiments with fixed budget. We consider the autoregressive Gaussian process model and the optimal nested design that maximizes the prediction accuracy subject to the budget constraint. An approximate solution is identified through the idea of multilevel approximation and recent error bounds of Gaussian process regression. The proposed (approximately) optimal designs admit a simple analytical form. We prove that, to achieve the same prediction accuracy, the proposed optimal multi-fidelity design requires much lower computational cost than any single-fidelity design in the asymptotic sense. The last work is proposed to model complex experiments when the distributions of training and testing input features are different (referred to as domain adaptation). In this work, we propose a novel transfer learning algorithm called Renewing Iterative Self-labeling Domain Adaptation (Re-ISDA) to tackle the domain adaptation problem. The learning problem is formulated as a dynamic programming model, and the latter is then solved by an efficient greedy algorithm by adding a renewing step to the original ISDA algorithm. This renewing step helps avoid a potential issue of the ISDA that the possible mis-labeled samples by a weak predictor in the initial stage of the iterative learning can cause serious harm to the subsequent learning process. Numerical studies show that the proposed method outperforms prevailing transfer learning methods. The proposed method also achieves high prediction accuracy for a cervical spine motion problem.