A Mathematical Optimization Review of the Federated Learning Model Aggregation Mechanism Incorporating Secure Multi-Party Computation

DOI: https://doi.org/10.62517/jbdc.202601208

Author(s)

Tianyi Su

Affiliation(s)

Chongqing Finance and Economics College, Chongqing, China

Abstract

This paper reviews the mathematical optimization methods of the federated learning model aggregation mechanism incorporating secure multi-party computation. First, the basic concepts and principles of federated learning and secure multi-party computation are introduced. Subsequently, the design and optimization strategies of the aggregation mechanism in the federated learning model are analyzed. Then, from the perspective of mathematical optimization, the optimization methods of different aggregation mechanisms are discussed in detail, including gradient aggregation, parameter aggregation, and model aggregation, etc. Finally, the existing methods are compared and summarized, and the future research directions are pointed out.

Keywords

Federated Learning; Secure Multi-Party Computation; Aggregation Mechanism; Mathematical Optimization; Model Aggregation

References

[1] McMahan B, Moore E, Ramage D, et al. Communication-efficient learning of deep networks from decentralized data[C]/Proceedings of the 20th International Conference on Artificial Intelligence and Statistics(AISTATS). PMLR, 2017: 1273-1282. [2] Tang Z, Shi S, Chu X, et al. On the convergence of communication-efficient local SGD for federated learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2021, 35(11): 9835-9843. [3] Li T, Sahu A K, Zaheer M, et al. Federated optimization in heterogeneous networks[C]//Proceedings of Machine Learning and Systems (MLSys). 2020, 2: 429-450. [4] Karimireddy S P, Kale S, Mohri M, et al. Scaffold: Stochastic controlled averaging for federated learning[C]//International Conference on Machine Learning (ICML). PMLR, 2020: 5132-5143. [5] Reddi S, Charles Z, Zaheer M, et al. Adaptive federated optimization[C]//International Conference on Learning Representations (ICLR). 2021. [6] Ji S, Pan S, Li G, et al. FedQS: A semi-asynchronous federated learning framework for non-IID data[J].IEEE Transactions on Knowledge and Data Engineering, 2023, 35(12): 12567-12582. [7] Xu J, Tong X, Jiang X, et al. FedAdaSS: Federated learning with adaptive parameter server selection forcommunication efficiency[J]. IEEE Internet of Things Journal, 2024, 11(5): 8234-8248. [8] Al-Shedivat M, Gillenwater J, Xing E, et al. Federated learning via posterior averaging: A Bayesianperspective[C]//International Conference on Learning Representations (ICLR). 2021. [9] Ji S, Jiang T, Wang Z, et al. Privacy preserved federated learning with attention-based aggregation[J]. arXivpreprint arXiv:2012.12073, 2020. [10] Chen F, Luo M, Dong Z, et al. Federated meta-learning with fast convergence and efficientcommunication[J]. arXiv preprint arXiv:1802.07876, 2018. [11] Shen T, Li Z, Zhang X, et al. FedMcon: An adaptive aggregation method for federated learning via metacontroller[C]//Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining.2024: 2543-2554. [12] Zhang Y, Liu R, Gupta S, et al. pFedGPA: Personalized federated learning via generative parameter aggregation with diffusion models[C]//Proceedings of the 38th Conference on Neural Information Processing Systems (NeurIPS). 2024. [13] Lin B Y, He S, Zeng Z, et al. FedNLP: A benchmarking framework for federated learning on natural language processing tasks[C]//Findings of the Association for Computational Linguistics: NAACL 2022. 2022:157-175. [14] Caldas S, Duddu S M K, Wu P, et al. LEAF: A benchmark for federated settings[J]. arXiv preprint arXiv:1812.01097, 2018. [15] Nguyen A L, Vu T D, Pham Q V, et al. A survey on federated learning for anomaly detection: Taxonomy, applications, and future directions[J]. IEEE Internet of Things Journal, 2024, 11(10): 16892-16915. [16] Wang J, Qi Z, Zhou M, et al. Distributed intrusion detection system based on federated learning for internetof things[C]//IEEE International Conference on Communications (ICC). 2022: 1-6. [17] Kirkpatrick J, Pascanu R, Rabinowitz N, et al. Overcoming catastrophic forgetting in neural networks[J]. Proceedings of the National Academy of Sciences (PNAS), 2017, 114(13): 3521-3526. [18] Wang J, Charles Z, Xu Z, et al. A field guide to federated optimization[J]. arXiv preprint arXiv:2107.06917,2021.