Otimização de hiperparâmetros do XGBoost utilizando meta-aprendizagem

Abstract

With computional evolution, there was a growth in the number of machine learning algorithms and in parallel, they became more complex and robust. With the growth of this complexity, it was increasingly necessary to focus on the configuration of algorithms, in accordance with their hyperparameters, with the aim of increasing the precision in the result of each one of them: which is not a trivial task. Thus, choosing the most suitable hyperparameters for a given data set can be a costly task both in terms of time and money. Thus, it is necessary that there are faster and more practical ways to find hyperparameters that will set up each algorithm individually. This work aims to use Meta-Learning as a viable solution for the recommendation of hyperparameters for the recent XGBoost machine learning algorithm, in order to reduce computional costs, also aiming at reducing costs for companies. This work used 198 datasets, following the idea of leave-one-out cross-validation for the experiments, thus making each of the datasets tested in the experiment against all others. In addition, were used 3 sets of Meta-Features available in the literature: general, statistical and info-theory for the characterization phase of each one of the datasets to compare the similarity among them. The experimental results attested the success of the application of the heuristics using Meta-Learning for their recommendation, Thus, initially a characterization of the data sets was made using three sets of Meta-Features, so that there was a way to compare the similarity between them and thus apply Meta-Learning to recommend the hyperparameters between the data sets used in the experiments. The results obtained were promising, making that in some cases, 86.36% of the tests had positive results, that is, the accuracy of XGBoost using Meta-Learning, had a better result than the standard hyperparemeters used by XGBoost in 86.36% of cases.