FedReLa: Imbalanced Federated Learning via Re-Labeling
Guangzheng Hu, Patricia Menéndez, Feng Liu, Mingming Gong, Guanghui Wang, Liuhua Peng
Read on arXiv →
Key claim
FedReLa improves accuracy in federated learning without global class knowledge.
In plain English
Imagine you're trying to train a machine learning model using data from many different users, but each user has a different amount of data and different types of data. This is a common scenario in federated learning, where privacy is key, but it leads to problems. When some classes of data are underrepresented, the model struggles to learn effectively, especially if the data is very unevenly distributed across users. This situation can cause the model to perform poorly, particularly on minority classes, which is what's called class imbalance and data heterogeneity.
To tackle these issues, the authors propose a method called FedReLa. Instead of needing to know the overall distribution of classes, FedReLa uses a feature-dependent approach to re-label samples. This means it can adjust the labels based on the features of the data, helping to correct the model's decision boundaries. The beauty of this method is that it can be integrated with existing algorithms without adding extra communication costs, which is crucial in federated learning scenarios.
In practical terms, this means that if you're building a system that relies on federated learning, you can expect better performance on minority classes and overall accuracy when using FedReLa. It significantly outperforms previous methods, making it a valuable tool for developers working with decentralized data.
The approach introduces a new method for addressing class imbalance and data heterogeneity in federated learning, which is a significant extension of existing techniques.
The claims are supported by extensive experiments demonstrating improvements over previous methods, indicating solid validation.
Deep reliability assessment
The experiments support that FedReLa can improve minority/tail and overall accuracy in simulated federated class-imbalance settings using standard vision datasets and Dirichlet non-IID partitioning. The broader claim that it is model-, data-, and deployment-agnostic is less fully established because the visible evidence is mostly image classification under synthetic imbalance rather than real federated deployments with noisy labels, partial participation, privacy mechanisms, or non-vision data.
Reproducibility
No code repository is mentioned in the provided abstract, introduction, results, or conclusion snippets. The paper evaluates on public datasets including Fashion-MNIST, CIFAR-10/100, and ImageNet-LT, with imbalance and client heterogeneity generated through described sampling procedures.
Key figure
Figure 1 shows FedReLa as a one-shot local preprocessing step at a chosen communication round, where each client uses the received global model to compute posterior probabilities, standardize them, re-label selected local samples, and then continue federated training on the modified local dataset.