Federated User Representation Learning
Duc Bui is a PhD candidate at Real-Time Computing Lab, working with Professor Kang Shin, at the Computer Science and Engineering Department, University of Michigan. His current research lies at the intersection of user privacy and natural language processing/machine learning. He previously researched and published papers on top-tier conferences in mobile systems and networks. He received Master of Science from KAIST, South Korea, in 2013 and Bachelor of Science from HUST, Vietnam, in 2010. He received awards/fellowships from Microsoft, Qualcomm, Korean Government and Vietnam Ministry of Information and Communications, and interned at Facebook, Google, Microsoft Research and Samsung.
Federated Learning (FL) protects user privacy by training machine learning models in a distributed and privacy-preserving way. While collaborative personalization, such as through learned user representations, can improve the prediction accuracy significantly, personalization in FL poses many challenges due to its distributed nature, high communication costs, and privacy constraints. In this talk, we present Federated User Representation Learning (FURL), a simple, scalable, privacy-preserving and resource-efficient way to utilize existing neural personalization techniques in the FL setting. We evaluate FURL on two datasets, demonstrating a significant improvement in model quality with 8% and 51% performance increases, and approximately the same level of performance as centralized training with only 0% and 4% reductions. Furthermore, we show that user embeddings learned in FL and the centralized setting have a very similar structure, indicating that FURL can learn collaboratively through the shared parameters while preserving user privacy.
Related seminars
Stony Brook University
Washington State University
University of Melbourne
National University of Singapore