Federated Learning Framework Over Wireless Edge Network: Optimizing Resources and Learning Algorithms

The extraordinary improvements in the Internet of Things (IoT), ubiquitous communications, and Artificial Intelligence (AI) have induced an exponential growth in the size of data generated every day by edge devices (e.g., IoT devices, smartphones, sensors, actuators). Leveraging the proliferation of AI, the generated data, and mobile edge computing (MEC) techniques can bring valuable innovative services to end-users. Lately, MEC techniques have gained considerable efforts from practitioners and researchers because of their potential in reducing latency and delivering an elegant quality of experience for end devices. It is envisioned that MEC will be an enabling tool for sixth-generation (6G) networks permitting new emerging applications, such as human-centric services, virtual reality, and augmented reality, consequently realizing the vision of network intelligence. Yet, transferring massive volumes of user data to a central server incurs unwanted communication costs and security risks due to the networks’ limitations, scalability issues, inadequate bandwidth, and, most importantly, users’ privacy. To satisfy these needs, Federated Learning (FL) has been recently introduced to train and update a shared model collaboratively without sharing the raw data and only sharing the model parameters, leading to preserving the user’s privacy and reducing the communication costs. FL algorithms have recently been pushed towards the network edge and in the development of Federated Edge Learning (FEEL) systems providing low latency edge intelligence. FEEL can be seen as a cutting-edge collaborative machine learning (ML) technique for future IoT and edge systems. However, there are still several challenges associated with deploying FEEL. First, current network resources (e.g., Bandwidth, Up-link and downlink speed, energy) are still becoming a bottleneck for faster learning due to the frequent computation and communication tasks, an abundance of energy expenses required for several rounds, resulting in degrading learning performance under the training time allowance. In addition, FEEL has critical challenges related to imbalanced, unlabeled, and non-i.i.d. (non-independent and identically distributed) data distribution amongst clients, leading to slower convergence and increasing the communication cost as the participant devices need a large number of rounds to converge. While many research efforts are devoted to tackling these challenges, there is a lack of designing efficient approaches for reliable FEEL systems, considering optimizing the selection and scheduling policies, learning algorithms, and the available communication computation edge network resources. This gap motivates us to propose novel approaches: (1) A joint participants selection and resource allocation scheme is proposed to minimize training time, increase data utilization, and provide efficient resource allocation. (2) Novel solutions for energy-efficient FEEL are proposed by jointly considering local training data, available computation, communications resources, and deadline constraints of FEEL rounds to reduce energy consumption. (3) A new client selection approach for Clustered Federated Learning (CFL) is introduced to minimize the training time and accelerate the convergence rate while tackling non-i.i.d and unbalanced data distribution problems to provide more reliable and specialized ML models. (4) Principal design aspects for enabling federated learning at the edge networks are presented, accounting for the problem of unlabeled data. We introduce a semi-supervised federated edge learning scheme called FedSem that leverages unlabeled data in real-time. (5) Novel solutions to run semi-supervised FL over wireless network edge are proposed, considering the limited resources and deadline constraints and realizing that unlabeled data can be automatically labeled during the training rounds to improve the performance of the global model.