Experimental evaluation of an automatic parameter setting system

Finding the parameter setting that will result in the optimal performance of a given algorithm for solving a problem is a tedious task. This paper briefly describes a system that automatically chooses the best algorithm parameter configuration conditioned by the current problem instance to solve. The system uses bayesian networks (BN) and case-based reasoning (CBR) methodology to find such a configuration. CBR provides a mechanism to acquire knowledge about the specific problem domain. BN provide a tool to model quantitative and qualitative relationships between parameters of interest. However, the aim of this work is to empirically evaluate the system described, using as an example the configuration of a genetic algorithm that solves the root identification problem. In this context, we report on several statistically guided experimental evaluations. The experimental results confirm the validity of the proposed system and its potential effectiveness for configuring algorithms.