Federated learning enables machine learning models to be trained across decentralized devices or servers without sharing raw data. The main types are categorized based on how data is distributed and how participants collaborate. Here’s a breakdown of the three primary approaches:
1. Horizontal Federated Learning (HFL): This type is used when participants have datasets with the same feature space but different sample IDs. For example, multiple hospitals might collect similar patient data (e.g., lab results, diagnoses) but serve different populations. In HFL, each participant trains a local model on their data, and a central server aggregates these models (e.g., using federated averaging, or FedAvg). The aggregated model is then redistributed for further training. This approach is common in scenarios like mobile keyboard prediction, where millions of users contribute to a shared model without exposing individual typing data. Key challenges include handling uneven data distribution and ensuring efficient communication between devices and the server.
2. Vertical Federated Learning (VFL): VFL applies when datasets share the same sample IDs but differ in features. For instance, a bank and an e-commerce company might both have data on the same customers—the bank holds credit scores, while the e-commerce platform tracks purchase history. Here, participants collaborate to train a model using overlapping samples but distinct features. Techniques like secure multi-party computation (MPC) or homomorphic encryption are often used to compute joint results without exposing raw data. A practical example is fraud detection, where combining financial and transactional data improves accuracy. VFL requires careful alignment of overlapping samples and secure protocols to preserve privacy.
3. Federated Transfer Learning (FTL): FTL addresses cases where datasets differ in both samples and features. For example, a retail company with limited customer data might collaborate with a larger retailer in a different region to improve a recommendation model. Transfer learning techniques, such as feature mapping or domain adaptation, allow knowledge from one domain (e.g., user behavior in Region A) to be applied to another (Region B). This approach is useful when data scarcity or regulatory constraints prevent direct collaboration. FTL emphasizes techniques like model fine-tuning or intermediate representation sharing to bridge gaps between non-overlapping datasets.
Each type addresses specific data distribution scenarios, balancing privacy, efficiency, and model performance. Choosing the right approach depends on the problem’s structure, data availability, and privacy requirements.