Open Access

ARTICLE

Federated Learning with Consistency Optimization Algorithms under Non-IID Data

Rui Wu¹, Yehong Li², Hongjie Guo^3,*, Gangqiang Hu³, Changjun Zhou^3,*, Qile Zou⁴

1 School of Software Engineering, Jiangxi University of Science and Technology, Nanchang, China
2 Jiangxi Provincial Key Laboratory of Multidimensional Intelligent Perception and Control, Jiangxi University of Science and Technology, Ganzhou, China
3 School of Computer Science and Technology, Zhejiang Normal University, Jinhua, China
4 Faculty of Science and Engineering, University of Nottingham Ningbo China, Ningbo, China

* Corresponding Authors: Hongjie Guo. Email: ; Changjun Zhou. Email:

Computers, Materials & Continua 2026, 89(1), 63 https://doi.org/10.32604/cmc.2026.083715

Received 09 April 2026; Accepted 03 July 2026; Issue published 13 August 2026

Abstract

Federated learning (FL) enables collaborative training of deep neural architectures while preserving data privacy, yet its performance often deteriorates in non-IID scenarios, which stems from client-side distribution drift and divergent local updates induced by pervasive data heterogeneity. This challenge is particularly critical for maintaining the structural consistency and generalization of neural models across diverse, distributed sources with significant distribution shifts. In this paper, we investigate how to effectively mitigate label distribution shift and feature distribution skew to enhance the global representation stability of neural architectures. We propose Federated Learning with Consistency Optimization Algorithms (FedCO), a novel optimization framework that incorporates a label-skew-aware correction loss and neural feature distribution regularization during local training. Specifically, our method aligns the internal statistics of architectural components between local and global models to suppress feature-space drift. Combined with an adaptive global aggregation mechanism guided by label entropy, this approach ensures that model updates from heterogeneous clients are consistently integrated into the global architectural parameters. Experimental results on multiple benchmarks demonstrate that FedCO significantly improves accuracy and convergence under diverse non-IID settings. For instance, on CIFAR-10 with extreme heterogeneity (α=0.05), FedCO achieves 73.64% accuracy, outperforming FedAvg by 7.32%; it also attains the highest accuracies on CIFAR-100 (62.63%) and TinyImageNet (37.82%) under the same setting. Our findings provide a robust strategy for integrating distribution-aware local training with adaptive structural aggregation, offering new insights into enhancing the reliability of distributed neural systems in real-world deployments. The code related to this algorithm is available at https://github.com/Donglin0730/FedCO.

---

Keywords

---