Metamodeling-based simulation optimization in manufacturing problems: a comparative study

In the context of modern industry, optimization emerges as one of the most powerful tools, allowing decision-makers to allocate their resources more assertively and deal with complex manufacturing problems. Moreover, manufacturing systems usually involve activities’ interdependency and high stochastic levels, which are necessary to associate optimization and simulation techniques to solve problems. Although simulation optimization is a powerful technique, it can converge on a good solution, which often limits its use in day-to-day operations. As an alternative, metamodels may be used to replace simulation models in the optimization process. In recent years, with the development in the machine learning area, algorithms with high learning capacity have emerged, making the metamodel-based simulation optimization (MBSO) a promising study field. Based on the latest theoretical research on the theme, MBSO techniques have been widely used to solve manufacturing problems. However, there is still no consensus about the experimental design, the learning algorithms, and the importance of the hyperparameter optimization step. Then, the article evaluates the performance of six machine learning algorithms trained with and without hyperparameter optimization, two experimental designs, and five different sample sizes to build metamodels in three real manufacturing cases. Based on the results, the random forest algorithm and the random design with 40 × sample size expressed the better performance to metamodels’ development. Furthermore, the hyperparameter optimization step reduced the metamodels’ error in about 32.83%.