A SURVEY ON CHALLENGES OF FEDERATED LEARNING

Details

Hits: 892

Volume 5 (2), December 2022, Pages 273-285

Aliyev S.I.

Azerbaijan State Oil and Industry University, Baku, Azernaijan, This email address is being protected from spambots. You need JavaScript enabled to view it.

Abstract

Federated Learning is a new paradigm of Machine Learning. The main idea behind FL is to provide a decentralized approach to Machine Learning. Traditional ML algorithms are trained in servers with data collected by clients, but data privacy is the primary concern. This is where FL comes into play: all clients train their model locally and then share it with a global model in the server and receive it back. However, FL has problems, such as possible cyberattacks, aggregating most appropriately, scaling the non-IID data, etc. This paper highlights current attacks, defenses, pros and cons of aggregating methods, and types of non-IID data based on publications in this field.

Keywords:

Federated learning, Challenges of FL, Aggregation methods in FL, Attacks and vulnerabilities, Defenses, non-iid data.

DOI: https://doi.org/10.32010/26166127.2022.5.2.273.285

Reference 

Al Hayajneh, A., Bhuiyan, M. Z. A., & McAndrew, I. (2020). Improving Internet of Things (IoT) security with software-defined networking (SDN). Computers, 9(1), 8.

Aono, Y., Hayashi, T., Wang, L., & Moriai, S. (2017). Privacy-preserving deep learning via additively homomorphic encryption. IEEE Transactions on Information Forensics and Security, 13(5), 1333-1345.

Benmalek, M., Benrekia, M. A., & Challal, Y. (2022). Security of Federated Learning: Attacks, Defensive Mechanisms, and Challenges. Revue des Sciences et Technologies de l’Information-Série RIA: Revue d’Intelligence Artificielle, 36(1), 49-59.

Bhagoji, A. N., Chakraborty, S., Mittal, P., & Calo, S. (2019, May). Analyzing federated learning through an adversarial lens. In International Conference on Machine Learning (pp. 634-643). PMLR.

Blanco-Justicia, A., Domingo-Ferrer, J., Martínez, S., Sánchez, D., Flanagan, A., & Tan, K. E. (2021). Achieving security and privacy in federated learning systems: Survey, research challenges and future directions. Engineering Applications of Artificial Intelligence, 106, 104468.

Cao, D., Chang, S., Lin, Z., Liu, G., & Sun, D. (2019, December). Understanding distributed poisoning attack in federated learning. In 2019 IEEE 25th International Conference on Parallel and Distributed Systems (ICPADS) (pp. 233-239). IEEE.

Chen, Y., Su, L., & Xu, J. (2017). Distributed statistical machine learning in adversarial settings: Byzantine gradient descent. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 1(2), 1-25.

Chen, Y., Sun, X., & Jin, Y. (2019). Communication-efficient federated deep learning with layer-wise asynchronous model update and temporally weighted aggregation. IEEE transactions on neural networks and learning systems, 31(10), 4229-4238.

Criado, M. F., Casado, F. E., Iglesias, R., Regueiro, C. V., & Barro, S. (2022). Non-IID data and Continual Learning processes in Federated Learning: A long road ahead. Information Fusion, 88, 263-280.

Ek, S., Portet, F., Lalanda, P., & Vega, G. (2020, September). Evaluation of federated learning aggregation algorithms: application to human activity recognition. In Adjunct Proceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2020 ACM International Symposium on Wearable Computers (pp. 638-643).

Gao, L., Fu, H., Li, L., Chen, Y., Xu, M., & Xu, C. Z. (2022). FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 10112-10121).

Garcia-Molina, H., Joglekar, M., Marcus, A., Parameswaran, A., & Verroios, V. (2016). Challenges in data crowdsourcing. IEEE Transactions on Knowledge and Data Engineering, 28(4), 901-911.

Gosselin, R., Vieu, L., Loukil, F., & Benoit, A. (2022). Privacy and Security in Federated Learning: A Survey. Applied Sciences, 12(19), 9901.

Hayes, J., & Ohrimenko, O. (2018). Contamination attacks and mitigation in multi-party machine learning. Advances in neural information processing systems, 31.

Kairouz, P., McMahan, H. B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A. N., ... & Zhao, S. (2021). Advances and open problems in federated learning. Foundations and Trends® in Machine Learning, 14(1–2), 1-210.

Li, D., & Wang, J. (2019). Fedmd: Heterogenous federated learning via model distillation. arXiv preprint arXiv:1910.03581.

Li, Q., Diao, Y., Chen, Q., & He, B. (2022, May). Federated learning on non-iid data silos: An experimental study. In 2022 IEEE 38th International Conference on Data Engineering (ICDE) (pp. 965-978). IEEE.

Li, T., Sahu, A. K., Talwalkar, A., & Smith, V. (2020). Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine, 37(3), 50-60.

Lin, J., Du, M., & Liu, J. (2019). Free-riders in federated learning: Attacks and defenses. arXiv preprint arXiv:1911.12560.

Liu, K., Dolan-Gavitt, B., & Garg, S. (2018, September). Fine-pruning: Defending against backdooring attacks on deep neural networks. In International Symposium on Research in Attacks, Intrusions, and Defenses (pp. 273-294). Springer, Cham.

Liu, P., Xu, X., & Wang, W. (2022). Threats, attacks and defenses to federated learning: issues, taxonomy and perspectives. Cybersecurity, 5(1), 1-19.

Lyu, L., Yu, H., & Yang, Q. (2020). Threats to federated learning: A survey. arXiv preprint arXiv:2003.02133.

Lyu, L., Yu, H., Ma, X., Chen, C., Sun, L., Zhao, J., ... & Philip, S. Y. (2022). Privacy and robustness in federated learning: Attacks and defenses. IEEE transactions on neural networks and learning systems.

Pan, S. J., & Yang, Q. (2010). A survey on transfer learning. IEEE Transactions on knowledge and data engineering, 22(10), 1345-1359.

Rodríguez-Barroso, N., Jiménez-López, D., Luzón, M. V., Herrera, F., & Martínez-Cámara, E. (2023). Survey on federated learning threats: concepts, taxonomy on attacks and defences, experimental study and challenges. Information Fusion, 90, 148-173.

Sannara, E. K., Portet, F., Lalanda, P., & German, V. E. G. A. (2021, March). A federated learning aggregation algorithm for pervasive computing: Evaluation and comparison. In 2021 IEEE International Conference on Pervasive Computing and Communications (PerCom) (pp. 1-10). IEEE.

Su, H., Maji, S., Kalogerakis, E., & Learned-Miller, E. (2015). Multi-view convolutional neural networks for 3d shape recognition. In Proceedings of the IEEE international conference on computer vision (pp. 945-953).

Sun, T., Li, D., & Wang, B. (2022). Decentralized federated averaging. IEEE Transactions on Pattern Analysis and Machine Intelligence.

Tolpegin, V., Truex, S., Gursoy, M. E., & Liu, L. (2020, September). Data poisoning attacks against federated learning systems. In European Symposium on Research in Computer Security (pp. 480-501). Springer, Cham.

Wang, D., Li, C., Wen, S., Nepal, S., & Xiang, Y. (2020). Man-in-the-middle attacks against machine learning classifiers via malicious generative models. IEEE Transactions on Dependable and Secure Computing, 18(5), 2074-2087.

Wang, H., Yurochkin, M., Sun, Y., Papailiopoulos, D., & Khazaeni, Y. (2020). Federated learning with matched averaging. arXiv preprint arXiv:2002.06440.

Wu, Q., He, K., & Chen, X. (2020). Personalized federated learning for intelligent IoT applications: A cloud-edge based framework. IEEE Open Journal of the Computer Society, 1, 35-44.

Yurochkin, M., Agarwal, M., Ghosh, S., Greenewald, K., Hoang, N., & Khazaeni, Y. (2019, May). Bayesian non-parametric federated learning of neural networks. In International Conference on Machine Learning (pp. 7252-7261). PMLR.

Zhang, J., Chen, B., Cheng, X., Binh, H. T. T., & Yu, S. (2020). Poisongan: Generative poisoning attacks against federated learning in edge computing systems. IEEE Internet of Things Journal, 8(5), 3310-3322.

Zhang, J., Zhu, H., Wang, F., Zhao, J., Xu, Q., & Li, H. (2022). Security and Privacy Threats to Federated Learning: Issues, Methods, and Challenges. Security and Communication Networks, 2022.

Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., & Chandra, V. (2018). Federated learning with non-iid data. arXiv preprint arXiv:1806.00582.

Zhu, H., Xu, J., Liu, S., & Jin, Y. (2021). Federated learning on non-IID data: A survey. Neurocomputing, 465, 371-390.

Zhu, Z., Hong, J., & Zhou, J. (2021, July). Data-free knowledge distillation for heterogeneous federated learning. In International Conference on Machine Learning (pp. 12878-12889). PMLR.