A novel tree-based dynamic heterogeneous ensemble method for credit scoring

•A tree-based heterogeneous ensemble credit scoring model is proposed.•Advanced GBDT-based methods function as components of our proposal.•An overfitting-cautious ensemble selection strategy is developed.•Our proposal outperforms the benchmark models significantly in most cases.•Our proposal is robust to slight modification on base model and fitness function. Ensemble models have been extensively applied to credit scoring. However, advanced tree-based classifiers have been seldom utilized as components of ensemble models. Moreover, few studies have considered dynamic ensemble selection. To fill the research gap, this paper aims to develop a novel tree-based overfitting-cautious heterogeneous ensemble model (i.e., OCHE) for credit scoring which departs from existing literature on base models and ensemble selection strategy. Regarding base models, tree-based techniques are employed to acquire a balance between predictive accuracy and computational cost. In terms of ensemble selection, the proposed method can assign weights to base models dynamically according to the overfitting measure. Validated on five public datasets, the proposed approach is compared with several popular benchmark models and selection strategies on predictive accuracy and computational cost measures. For predictive accuracy, the proposed approach outperforms the benchmark models significantly in most cases based on the non-parametric significance test. It also performs marginally better than several state-of-the-art studies. Our proposal remains robust in several scenarios. In terms of computational cost, the proposed method provides acceptable performance and benefits from GPU acceleration considerably.