Synergetic Focal Loss for Imbalanced Classification in Federated XGBoost

Applying sparsity- and overfitting-aware eXtreme Gradient Boosting (XGBoost) for classification in federated learning allows many participants to train a series of trees collaboratively. Since various local multiclass distributions and global aggregation diversity, model performance plummets as convergence slowly and accuracy decreases. Worse still, neither the participants nor the server can detect this problem and make timely adjustments. In this article, we provide a new local-global class imbalance inconsistency quantification and utilize softmax as the activation and focal loss, a dynamically scaled cross-entropy loss, in federated XGBoost to mitigate local class imbalance. Moreover, we propose a simple but effective hyperparameter determination strategy based on local data distribution to adjust the sample weights among noncommunicating participants, synergetic focal loss, to solve the inconsistency of local and global class imbalance, a unique characteristic of federated learning. This strategy is perfectly integrated into the original classification algorithm. It requires no additional detectors or information transmission. Furthermore, a dynamical for loop is designed to capture an optimum hyperparameter combination. Finally, we conduct comprehensive tabular- and image-based experiments to show that synergetic focal loss used in federated XGBoost achieves faster convergency and significant accuracy improvement. Simulation results prove the effectiveness of the proposed principle of configuring sample weights.