Efficient Exploitation of Multiple Information Sources in Multi-Objective Constrained Bayesian Optimization
Loading...
Date
2022-09-20
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
With the technological advancements made in recent years, more powerful computational machines are built. The ability to handle large computational problems, both in terms of storage requirements and processing time has brought the opportunity to simulate complex systems more accurately than ever. Engineers, in particular, are interested to take advantage of using advanced computational machines for their design and modeling purposes. Although powerful computers can model complex systems to predict outcomes, budget allocations remain an issue in design and optimization tasks. Running large computational problems, simulations or experiments can be cost-prohibitive in terms of time or computational resources, for example, when highly accurate and reliable results are expected. Therefore, it is desired to spend available computational resources wisely toward a design goal to avoid running futile calculations.
Recently, Bayesian optimization techniques have been developed and are widely employed to solve design problems in many engineering fields. The popularity of Bayesian optimization frameworks comes from the fact that they are able to work with minimal information and use a heuristic-based search strategy to probe the design space, looking for potentially informative experiments about optimal designs. Most of the Bayesian optimization frameworks are developed to use a single model to collect mapping information from design space to objective space and find the optimal design region, if not exactly a point. However, in cases of high-dimensional design spaces or complex objective functions, these frameworks still need to evaluate a large number of designs that can be costly and almost not practical considering computational resource limitations. Simplified models have been developed to lower the expenses related to evaluations of complex models but this comes at the price of accuracy loss. A simplified model has a lower fidelity but is able to give estimations of a quantity of interest at a much lower cost. The potential here is to employ several of these simplified models to obtain as much information as possible regarding an expensive complex model, known as the ground truth, by fusing the models and considering the correlation between the models and the ground truth. This approach opens up new ideas about extending Bayesian optimization frameworks to employ multiple sources of information instead of relying on a single expensive model to achieve a less costly design process.
Although several approaches exist to fuse different sources in multifidelity Bayesian optimization frameworks, there are not many of them developed to deal with multi-objective functions. In many engineering design problems, there are multiple quantities of interest to consider when looking for the optimal design. The issue here is the solution to such design problems is not a single design but a set of designs to be discovered. This can be computationally demanding owing to the fact it is searching for a region instead of a single point.
Another issue with Bayesian optimization frameworks is that they tend to underperform when the dimensionality of the design space increases. While statistical techniques can be employed to define the most important design variables and discard the remaining unimportant ones to reduce the dimensionality of the design space, it comes at the price of losing accuracy. Also, it may be misleading since there is a possibility of having the optimal design laid somewhere close to the regions ignored to be searched.
The next problem is, most often, there are several design constraints in an engineering design problem. It is crucial to know the constraints and recognize feasible regions before spending re-sources to search the design space. Sometimes, the models representing the constraints are also expensive to query and it is computationally prohibitive to sample a large number of points to identify the feasible region boundaries. Additionally, in some problems, a constraint can be in form of a binary check to see if a condition is satisfied or not. In such cases, there is not a continuous function to be modeled and check if its values lie within specific bounds. Additionally, the uncertainty in the feasibility prediction of a design is important in decision making processes once we know some constraints are known as hard constraints and they should be treated more conservatively.
In this study, we aim to address the issues mentioned above: first, we develop a multifidelity Bayesian optimization framework suitable to optimize multi-objective functions. The expected hypervolume improvement is employed as the criterion to look for optimal design regions while balancing the exploitation of the present system’s information and the exploration of new regions in the design space. Second, we propose a novel framework to use an adaptive active subspace method to efficiently recognize the important directions in the design space to form a lower dimensional space and reduce the dimensionality of the problem while it still allows searching all design variables but in different degrees. Finally, a multifidelity Bayesian classification framework is proposed to be employed within an optimization framework to solve constrained optimization problems more efficiently by actively learning the feasible region boundaries beforehand. Shannon Entropy definition is used to quantitatively determine the uncertainty, which is usually larger closer to the boundaries. We show the performance of our proposed frameworks on different test problems and then, its application to some engineering problems.
Description
Keywords
Bayesian Optimization, Multi-fidelity Approach, Bayesian Classification, Dimensionality Reduction, Active Subspace