[Upcoming paper]
In the Natural Language Processing (NLP) domain, zero-shot learning has become a mainstream paradigm to exploit large pre-trained models, such as GPT-3, T5 and BART in various tasks without any supervised label. However, fine-tuning these models on a few samples in a few-shot setting is still useful to adapt to a specific user or task. We propose to leverage few-shot learning of state-of-the-art NLP models with federated learning with the objective of pooling together several such specific fine-tuned models into a single central model that is better than any individual model, without having access to private local data. Our experiments are carried out using both an Induction Network and a pretrained BART model for sentiment analysis in English on the Amazon Review benchmark.
supervised-project-nlp
├── Bibliography
├── Paper
└── Realisation
├── DiverseFewShot_Amazon
├── bart model
├── federated transfer model
├── induction network
└── models