Acquisition-Guided Sampling-Based Approach for Detection of Regions of Interest

Abstract

Bayesian optimization (BO), a sequential design strategy for global optimization problem, has gained popularity during last decades for its capability of handling computationally expensive derivative-free objective functions with the adoption of a probabilistic framework. Its sample-efficiency property facilitates learning under data scarcity. There are two essential components involved in BO, namely the surrogate statistical model and acquisition function. Surrogate statistical model is designed to capture the behavior of the target objective function, while acquisition function plays an important role in providing a measurement for the goodness of suggested specifications and balancing exploration and exploitation.

This study is motivated by the detection of region of interest in circuit design and hyperparameter tuning in machine learning. With the rapid development in technology and science, the specification for circuit design can be extremely complicated, the evaluation of which in practice is expensive and heavily dependent on manual effort. The need of faster failure/success detection within a limited budget and better coverage of possible cases of interest for product quality control emerges. Similar circumstances can be found in hyperparameter tuning in machine learning. The growing complexity in model design also calls for better automatic tuning approaches instead of relying on heuristic choices.

In the first part of this study, we propose a new sampling-based approach with probability distribution specified by the acquisition functions based on the framework of Bayesian optimization. Optimization and Monte Carlo sampling are two computational strategies that enable statistical learning and decision making. Under certain conditions, sampling can be more efficient in terms of computation. Our proposed acquisition-guided sampling-based approach is capable of providing flexibility in multiple candidate suggestion, strong exploration over the space and incorporation with other criteria in the determination of the next evaluation point.

For the second part of this research, we develop a few new acquisition functions tailored to our problem. In particular, we propose two modifications of the existing acquisition functions. First, information of pre-specified threshold is introduced to existing acquisition functions, since region detection rather than global optimization is the primary goal. After shifting the distribution mass to areas with performance values close to the desired threshold, the algorithm is able to sample more specifications near ROIs. Second, the congestion at a local region may result in redundant computation with limited information gain. To solve this problem, an adjusted distance term is designed to encourage the wide spread of points by altering the magnitudes of acquisition values. Moreover, to better explore susceptible regions, we develop a new acquisition function called probability ratio (PR) to enlarge the difference between probabilities assigned to points that we want to pay more attention to and those we have no interest in. For all newly proposed acquisition functions, illustrative numerical results are consistent with our principles of designs, demonstrating that they have potential in applications such as automated circuit design and verification, as well as hyperparameter tuning in machine learning.