-
Notifications
You must be signed in to change notification settings - Fork 3
Repository that contains the code for the paper titled, 'Unifying Distillation with Personalization in Federated Learning'.
License
siddharthdivi/Unifying-Distillation-with-Personalization-in-Federated-Learning
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
########################################################################################################## UNIFYING DISTILLATION WITH PERSONALIZATION IN FEDERATED LEARNING ########################################################################################################## # Unifying-Distillation-with-Personalization-in-Federated-Learning Repository that contains the code for the paper titled, 'Unifying Distillation with Personalization in Federated Learning'. NOTE: The data required to run the experiments is available at https://tinyurl.com/1hp9ywfa. -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- SECTION-1: SETTING UP THE ENVIRONMENT. -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- * Instructions to replicate the experiments in the paper. * Organisation of this directory. - Unifying Distillation with Personalization/ in Federated Learning | | | | - Personalized-Federated-Learning.tar.gz | | | - Personalised-Federated-Learning-data.tar.gz | | | - pFedMe.tar.gz | | | - README.txt (this file.) * All the folders with the associated code and the data have been compressed into tar files. * First step is to untar all the tar files. * Once the files have been decompressed, next create a new folder in the same level as the other folders and name it 'Personalised-Federated-Learning-results' (this is the folder where all the results will be written to). * Inside 'Personalised-Federated-Learning-results', create the following folders with the below given organisational structure: config/, EpochResults/, log/, results/, state_dict/. -Personalized-Federated-Learning-results/ | | | - config/ | | | - EpochResults/ | | | - log/ | | | - results/ | | | - state_dict/ * Then, go to the folder 'Personalized-Federated-Learning/Personalized_Federated/code/cifar/' and read the file named 'experiments_Replication.txt' to generate the experimental results on CIFAR-10. * After this, go to the folder 'Personalized-Federated-Learning/Personalized_Federated/code/mnist/' and read the file named 'experiments_Replication.txt' to generate the experimental results on MNIST. * Once the previous two steps are complete, then go to the folder 'pFedMe'. <> If the results/ directory is not already existing, then create a new results/ directory. <> Then, read the experiments_Replication.txt in pFedMe/ to understand how to generate the experimental results for CIFAR-10 and MNIST for pFedMe and Per-FedAvg. -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- SECTION-2: CONDUCTING EXPERIMENTS ON CIFAR-10 AND MNIST FOR PersFL-KD, FedAvg, FedPer, PersFL-GD -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- * Running the experiments on CIFAR-10. <> cd to ~/Personalized-Federated-Learning/Personalized_Federated/code/cifar/. <> Run the following commands on the terminal: (1) To generate the results for the FedAvg model on CIFAR-10 nohup bash FedAvg.sh > FedAvg.out & (2) To generate the results for the FedPer model on CIFAR-10 nohup bash FedPer.sh > FedPer.out & (3) To generate the results for the PersFL model on CIFAR-10 nohup bash PersFL-KD.sh > PersFL-KD.out & (4) To generate the results for the variant of PersFL model on CIFAR-10 nohup bash PersFL-GD.sh > PersFL-GD.out & (5) To generate the results for Table 4: Opt teachers vs FedAvg model as teacher model for distillation experiment. nohup bash PersFL-KD-GlobInit.sh > PersFL-KD-GlobInit.out & * Running the experiments on MNIST. <> cd to ~/Personalized-Federated-Learning/Personalized_Federated/code/mnist/. <> Run the following commands on the terminal: (1) To generate the results for the FedAvg model on MNIST nohup bash FedAvg.sh > FedAvg.out & (2) To generate the results for the FedPer model on MNIST nohup bash FedPer.sh > FedPer.out & (3) To generate the results for the PersFL model on MNIST nohup bash PersFL-KD.sh > PersFL-KD.out & (4) To generate the results for the variant of PersFL model on MNIST nohup bash PersFL-GD.sh > PersFL-GD.out & (5) To generate the results for Table 5: Opt teachers vs FedAvg model as teacher model for distillation experiment. nohup bash PersFL-KD-GlobInit.sh > PersFL-KD-GlobInit.out & * Once these experiments are done running, the results for these experiments will be stored in ./EpochResults/ folder as pickle (pkl) files. -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- SUB-SECTION-2.1: ANALYSIS OF THE EXPERIMENTAL RESULTS ON CIFAR-10 AND MNIST FOR PersFL-KD, FedAvg, FedPer, PersFL-GD -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- * For an analysis of these results, please take a look at the Jupyter notebooks under the folder: '~/Personalized-Federated-Learning/Personalized_Federated/code/cifar/experiments_Replication/' for CIFAR-10 and under '~/Personalized-Federated-Learning/Personalized_Federated/code/cifar/experiments_Replication/' for MNIST. -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- SECTION-3: CONDUCTING EXPERIMENTS ON CIFAR-10 AND MNIST FOR pFedMe AND Per-FedAvg -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- * Running the experiments on CIFAR-10 & MNIST. <> cd to ~/pFedMe/. <> Run the following commands on the terminal: (1) To generate the results for the pFedMe model on CIFAR-10 nohup bash pFedMe_CIFAR-10.sh > pFedMe_CIFAR-10.out & (2) To generate the results for the Per-FedAvg model on CIFAR-10 nohup bash PerFed_CIFAR-10.sh > PerFed_CIFAR-10.out & (3) To generate the results for the pFedMe model on MNIST nohup bash pFedMe_MNIST.sh > pFedMe_MNIST.out & (4) To generate the results for the Per-FedAvg model on MNIST nohup bash PerFed_MNIST.sh > PerFed_MNIST.out & * Once these experiments are done running, the results for these experiments will be stored in ./results/ folder. -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- SUB-SECTION-3.1: ANALYSIS OF THE EXPERIMENTAL RESULTS ON CIFAR-10 AND MNIST FOR pFedMe AND Per-FedAvg -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- * For an analysis of these results, please take a look at the Jupyter notebooks under the folder: '~/pFedMe/experiments_Replication/'. ----- NOTE: ----- * We conduct our experiments on top of the following publicly available code-bases: <> Federated Learning with Personalization Layers: https://bit.ly/35dKebE <> Federated Adaptation (to generate the Data-split strategy 2): https://github.com/ebagdasa/federated_adaptation <> pFedMe (Personalized Federated Learning with Moreau Envelopes): https://github.com/CharlieDinh/pFedMe ----------------------------------------------------------------------------------------------------------------------[EOF]---------------------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------------------[EOF]----------------------------------------------------------------------------------------------------------------------
About
Repository that contains the code for the paper titled, 'Unifying Distillation with Personalization in Federated Learning'.
Topics
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published