Transformer-Based Federated Learning Models for Recommendation Systems

In today's data-driven environment, safeguarding user privacy is a top priority, particularly in machine learning applications. Our study introduces an innovative approach that combines the privacy-preserving attributes of federated learning with the advanced capabilities of transformer-based models, specifically tailored for recommendation systems. Federated learning emerges as a decentralized alternative to traditional machine learning, enhancing both user privacy and data security. Our research employs two distinct transformer models: BERT (Bidirectional Encoder Representations from Transformers) and BST (Behavior Sequence Transformer), within a federated learning context. The models performance is analyzed using the Amazon Customer Review and movielens-1m datasets. The empirical results are compelling: the federated BERT model achieves a notable 87% and 76% accuracy in the global model for 2 different datasets. Similarly, the federated BST model demonstrates a performance with an mean absolute error of 0.8. This research not only highlights the effectiveness of federated learning in boosting model accuracy but also emphasizes its crucial role in preserving user privacy. Our findings illustrate that integrating federated learning can lead to enhanced performance in recommendation systems without sacrificing data privacy. Consequently, this research marks a significant step forward in developing more effective, privacy-conscious machine learning solutions, contributing to the broader field of ethical and responsible AI.