gitabcworld / fewshotlearning Goto Github PK

View Code? Open in Web Editor NEW

254.0 7.0 57.0 33 KB

Pytorch implementation of the paper "Optimization as a Model for Few-Shot Learning"

License: MIT License

Python 100.00%

optimization few-shot python pytorch optimization-algorithms

fewshotlearning's Introduction

Optimization as a Model for Few-Shot Learning

This repo provides a Pytorch implementation for the Optimization as a Model for Few-Shot Learning paper.

Installation of pytorch

The experiments needs installing Pytorch

Data

For the miniImageNet you need to download the ImageNet dataset and execute the script utils.create_miniImagenet.py changing the lines:

pathImageNet = '<path_to_downloaded_ImageNet>/ILSVRC2012_img_train'
pathminiImageNet = '<path_to_save_MiniImageNet>/miniImagenet/'

And also change the main file option.py line or pass it by command line arguments:

parser.add_argument('--dataroot', type=str, default='<path_to_save_MiniImageNet>/miniImagenet/',help='path to dataset')

Installation

$ pip install -r requirements.txt
$ python main.py

Acknowledgements

Special thanks to @sachinravi14 for their Torch implementation. I intend to replicate their code using Pytorch. More details at https://github.com/twitter/meta-learning-lstm

Cite

@inproceedings{Sachin2017,
  title={Optimization as a model for few-shot learning},
  author={Ravi, Sachin and Larochelle, Hugo},
  booktitle={In International Conference on Learning Representations (ICLR)},
  year={2017}
}

Authors

Albert Berenguel (@aberenguel) Webpage

fewshotlearning's People

Contributors

Stargazers

Watchers

Forkers

ishanic ml-lab shubhampachori12110095 haofusheng codegank taihulight lim0606 hyzcn murari023 skmkedar changebio amwons mrsjlw gclouding dsp6414 b2220333 sxjscience qinxiaojinla hli2020 seindlut xiaodan3 bigdatasciencegroup itbeyond1230 fendaq xjtushujun ricestudygroup bai-yeqi caihengyu520 yuzhiw michaelbeechan wpfhtl yuanwanglll jumperwang crashmoon snowstu hins zialiu sailfish009 aegon007 hgoyal5 nan2020 kanglicheng chazard7 swimmingswam mymuli killerray yangnedu362 few-shot-learning-research future-superstar whitae minzchan yanyuefan ibagur ysy2000 onlysixpence huzehua97 mklee1089

fewshotlearning's Issues

Details about current situation ?

Did you solve the problem about the gradients not flowing to LSTM modules?

I was trying to implement Learning to Learn by gradient decent by gradient decent using tensorflow, and I encounter a similar problem, but I solved the problem by updateing cross-entropy loss in every step using newest weights of classifier(optimizee) , could this be a potential problem?

CUDA out of memory

Traceback (most recent call last): File "main.py", line 45, in <module> results = importlib.import_module(opt['metaLearner']).run(opt,data) File "/home/user/myproject/FewShotLearning/model/lstm/train-lstm.py", line 123, in run opt['batchSize'][opt['nTrainShot']]) File "/home/user/anaconda3/envs/FewShotLearning/lib/python2.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/home/user/myproject/FewShotLearning/model/lstm/metaLearner.py", line 149, in forward output, loss = learner(testInput, testTarget) File "/home/user/anaconda3/envs/FewShotLearning/lib/python2.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/home/user/myproject/FewShotLearning/model/lstm/learner.py", line 51, in forward output = self.modelF.net(inputs) File "/home/user/anaconda3/envs/FewShotLearning/lib/python2.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/home/user/myproject/FewShotLearning/model/lstm-classifier.py", line 79, in forward x = self.layer2(x) File "/home/user/anaconda3/envs/FewShotLearning/lib/python2.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/home/user/anaconda3/envs/FewShotLearning/lib/python2.7/site-packages/torch/nn/modules/container.py", line 92, in forward input = module(input) File "/home/user/anaconda3/envs/FewShotLearning/lib/python2.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/home/user/anaconda3/envs/FewShotLearning/lib/python2.7/site-packages/torch/nn/modules/batchnorm.py", line 76, in forward exponential_average_factor, self.eps) File "/home/user/anaconda3/envs/FewShotLearning/lib/python2.7/site-packages/torch/nn/functional.py", line 1623, in batch_norm training, momentum, eps, torch.backends.cudnn.enabled RuntimeError: CUDA out of memory. Tried to allocate 16.25 MiB (GPU 0; 11.91 GiB total capacity; 8.65 GiB already allocated; 17.06 MiB free; 950.50 MiB cached)
` nvidia-smi
Wed Apr 3 20:53:46 2019
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 396.26 Driver Version: 396.26 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 TITAN X (Pascal) Off | 00000000:02:00.0 Off | N/A |
| 23% 35C P8 16W / 250W | 1035MiB / 12196MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 1 TITAN X (Pascal) Off | 00000000:03:00.0 Off | N/A |
| 23% 36C P8 18W / 250W | 10MiB / 12196MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 2 TITAN X (Pascal) Off | 00000000:82:00.0 Off | N/A |
| 23% 30C P8 17W / 250W | 10MiB / 12196MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 3 TITAN X (Pascal) Off | 00000000:83:00.0 Off | N/A |
| 23% 33C P8 17W / 250W | 10MiB / 12196MiB | 0% Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 10217 C ...qun/anaconda2/envs/pt04_py27/bin/python 1021MiB |
+-----------------------------------------------------------------------------+
`

pytorch version?

Traceback (most recent call last):
File "main.py", line 39, in
results = importlib.import_module(opt['metaLearner']).run(opt,data)
File "/home/user/myproject/FewShotLearning/model/lstm/train-lstm.py", line 121, in run
opt['batchSize'][opt['nTrainShot']])
File "/home/user/anaconda3/envs/FewShotLearning/lib/python2.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/user/myproject/FewShotLearning/model/lstm/metaLearner.py", line 145, in forward
self.lstm2_fS_iS_cS_deltaS)
File "/home/user/anaconda3/envs/FewShotLearning/lib/python2.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/user/myproject/FewShotLearning/model/lstm/metalstm.py", line 166, in forward
fS_n = torch.stack(fS_n, 0)
TypeError: stack(): argument 'tensors' (position 1) must be tuple of Tensors, not Tensor

RuntimeError: One of the differentiated Variables appears to not have been used in the graph

Learner nParams: 32901
Traceback (most recent call last):
File "main.py", line 38, in
results = importlib.import_module(opt['metaLearner']).run(opt, data)
File "/home/i/meta/FewShotLearning/model/lstm/train-lstm.py", line 121, in run
opt['batchSize'][opt['nTrainShot']])
File "/home/i/conda/envs/py27/lib/python2.7/site-packages/torch/nn/modules/module.py", line 325, in call
result = self.forward(*input, **kwargs)
File "/home/i/meta/FewShotLearning/model/lstm/metaLearner.py", line 174, in forward
torch.autograd.grad(loss, self.lstm2.parameters())
File "/home/i/conda/envs/py27/lib/python2.7/site-packages/torch/autograd/init.py", line 158, in grad
inputs, only_inputs, allow_unused)
RuntimeError: One of the differentiated Variables appears to not have been used in the graph

I run your code with py27 env. However the error occurs and I don't know what goes wrong?

I thought there should be a '/' before the second '*' in line 50 of file 'create_miniImagenet.py'

Miniimagenet files

Hello,

Thank you for sharing the code. Now I am preparing for the reproduction of your code using miniimagenet. I am wondering if you could provide me the miniimagenet dataset. I would be very grateful for that.

Thank you very much!
[email protected]

Where is the file requirements.txt?

Cannot take a larger sample than population when 'replace=False'

Hello
I have few questions , maybe you could give me some advice.
What's the mean of samples_per_class and samples_per_class_eval in miniImagenet.py
And I want to used my own data in this program for Binary classification, which parts of the code should I have to alter?

How to extract miniimagenet from imagenet train dataset downloaded in a folder?

I have Imagenet 2012 train dataset downloaded and all the tars are extracted together in a single folder. How do get miniimagenet out of it? There are no zero-padded image names in the original dataset.