Heterogeneity-Guided Client Sampling: Towards Fast and Efficient Non-IID Federated Learning
Statistical heterogeneity of data present at client devices in a federated learning (FL) system renders the training of a global model in such systems difficult. Particularly challenging are the settings where due to communication resource constraints only a small fraction of clients can participate...
Gespeichert in:
Hauptverfasser: | , |
---|---|
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | Statistical heterogeneity of data present at client devices in a federated
learning (FL) system renders the training of a global model in such systems
difficult. Particularly challenging are the settings where due to communication
resource constraints only a small fraction of clients can participate in any
given round of FL. Recent approaches to training a global model in FL systems
with non-IID data have focused on developing client selection methods that aim
to sample clients with more informative updates of the model. However, existing
client selection techniques either introduce significant computation overhead
or perform well only in the scenarios where clients have data with similar
heterogeneity profiles. In this paper, we propose HiCS-FL (Federated Learning
via Hierarchical Clustered Sampling), a novel client selection method in which
the server estimates statistical heterogeneity of a client's data using the
client's update of the network's output layer and relies on this information to
cluster and sample the clients. We analyze the ability of the proposed
techniques to compare heterogeneity of different datasets, and characterize
convergence of the training process that deploys the introduced client
selection method. Extensive experimental results demonstrate that in non-IID
settings HiCS-FL achieves faster convergence than state-of-the-art FL client
selection schemes. Notably, HiCS-FL drastically reduces computation cost
compared to existing selection schemes and is adaptable to different
heterogeneity scenarios. |
---|---|
DOI: | 10.48550/arxiv.2310.00198 |