Federated Transfer Learning (FTL) is a hybrid learning paradigm that combines the principles of Federated Learning (FL) with those of Transfer Learning (TL) to effectively utilize distributed, heterogeneous data sources.
Formally, FTL consists of a set of clients \( \{C_i\}_{i=1}^N \), each possessing local data \( D_i \), which may lie in different but overlapping feature or data spaces.
The goal is to train a global model \( M \) that is created by aggregating locally trained models \( M_i \) or their representations, where transfer mechanisms enable knowledge transfer between different domains or feature spaces.
Typically, techniques such as feature mapping, domain adaptation, or model fine-tuning are employed to address the heterogeneity of the data.
FTL addresses challenges such as privacy, data heterogeneity, and limited local data quantities.
Definition:
“Federated Transfer Learning is a distributed learning procedure that transfers and aggregates model or representation knowledge between heterogeneous clients to produce a common model while keeping local data private.”
Source:
Yang, Q., Liu, Y., Chen, T., & Tong, Y. (2019). Federated Machine Learning: Concept and Applications. ACM Transactions on Intelligent Systems and Technology (TIST), 10(2), 12.