learngene's People
Contributors
Stargazers
Watchers
learngene's Issues
FileNotFoundError: [Errno 2] No such file or directory: './imagenet_exp/outputs/i_inheritable_random_trainnum200_wEWC/i_inheritable_cifar100_wEWC.xls'
I got some problems when I want to reconstruct individual model by using this command first time:
你好，我想按照readme的指示做个体模型重建，使用inheritable_cifar100_wEWC.py。
python inheritable_cifar100_wEWC.py --batch_size 32 --epochs 30 --num_works 21 --num_works_tt 5 --num_imgs_per_cat_train 10 --path /home/yupeng/learngene/learngene/exp_data/data_cifar100/2022-03-06_22:07:50/inheritabledataset/
Here is my log:
Data loading...
Model constructing...
0-layer:Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
1-layer:BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
2-layer:ReLU(inplace=True)
3-layer:Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
4-layer:BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
5-layer:ReLU(inplace=True)
6-layer:MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
7-layer:Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
8-layer:BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
9-layer:ReLU(inplace=True)
10-layer:Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
11-layer:BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
12-layer:ReLU(inplace=True)
13-layer:MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
......
......
......
6-layer:Linear(in_features=1024, out_features=64, bias=True)
7-layer:Softmax(dim=None)
Building snapshot file: ./imagenet_exp/outputs/i_inheritable_cifar100_wEWC/record_2022-03-13_23:39:37/Task_20
LL_Task 20 finished !
TT_Task 0 begins !
/home/yupeng/anaconda3/envs/learngene/lib/python3.9/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at /pytorch/c10/core/TensorImpl.h:1156.)
return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
=> Estimating diagonals of the fisher information matrix...
Estimate the fisher information of the parameters:
one
next:
Sun Mar 13 15:39:42 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.42.01 Driver Version: 470.42.01 CUDA Version: 11.4 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... On | 00000000:8A:00.0 Off | N/A |
| 32% 30C P8 29W / 350W | 1MiB / 24268MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 1 NVIDIA GeForce ... On | 00000000:8B:00.0 Off | N/A |
| 33% 30C P8 18W / 350W | 1MiB / 24268MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 2 NVIDIA GeForce ... On | 00000000:8C:00.0 Off | N/A |
| 32% 29C P8 28W / 350W | 1MiB / 24268MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 3 NVIDIA GeForce ... On | 00000000:DA:00.0 Off | N/A |
| 33% 30C P8 25W / 350W | 1MiB / 24268MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 4 NVIDIA GeForce ... On | 00000000:DB:00.0 Off | N/A |
| 33% 34C P2 110W / 350W | 3484MiB / 24268MiB | 11% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 5 NVIDIA GeForce ... On | 00000000:DC:00.0 Off | N/A |
| 33% 30C P8 20W / 350W | 3MiB / 24268MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 6 NVIDIA GeForce ... On | 00000000:DD:00.0 Off | N/A |
| 40% 29C P8 27W / 350W | 3MiB / 24268MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 7 NVIDIA GeForce ... On | 00000000:DE:00.0 Off | N/A |
| 39% 29C P8 23W / 350W | 3MiB / 24268MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 4 N/A N/A 15661 C python 3481MiB |
+-----------------------------------------------------------------------------+
Done!
Epoch: [0](test)[16/16] Time 0.014 ( 0.018) Data 0.009 ( 0.015) Acc@1 20.00 ( 20.00) Acc@5 100.00 (100.00)
=> Estimating diagonals of the fisher information matrix...
Estimate the fisher information of the parameters:
one
next:
Sun Mar 13 15:39:43 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.42.01 Driver Version: 470.42.01 CUDA Version: 11.4 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... On | 00000000:8A:00.0 Off | N/A |
| 32% 30C P8 29W / 350W | 1MiB / 24268MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 1 NVIDIA GeForce ... On | 00000000:8B:00.0 Off | N/A |
| 33% 30C P8 18W / 350W | 1MiB / 24268MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 2 NVIDIA GeForce ... On | 00000000:8C:00.0 Off | N/A |
| 32% 29C P8 28W / 350W | 1MiB / 24268MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 3 NVIDIA GeForce ... On | 00000000:DA:00.0 Off | N/A |
| 33% 30C P8 25W / 350W | 1MiB / 24268MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 4 NVIDIA GeForce ... On | 00000000:DB:00.0 Off | N/A |
| 33% 34C P2 126W / 350W | 3596MiB / 24268MiB | 40% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 5 NVIDIA GeForce ... On | 00000000:DC:00.0 Off | N/A |
| 33% 30C P8 20W / 350W | 3MiB / 24268MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 6 NVIDIA GeForce ... On | 00000000:DD:00.0 Off | N/A |
| 39% 29C P8 27W / 350W | 3MiB / 24268MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 7 NVIDIA GeForce ... On | 00000000:DE:00.0 Off | N/A |
| 39% 29C P8 23W / 350W | 3MiB / 24268MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 4 N/A N/A 15661 C python 3593MiB |
+-----------------------------------------------------------------------------+
Done!
Epoch: [1](test)[16/16] Time 0.013 ( 0.018) Data 0.010 ( 0.016) Acc@1 15.00 ( 20.00) Acc@5 100.00 (100.00)
=> Estimating diagonals of the fisher information matrix...
Estimate the fisher information of the parameters:
one
next:
......
......
......
Done!
Epoch: [28](test)[16/16] Time 0.013 ( 0.017) Data 0.010 ( 0.015) Acc@1 55.00 ( 52.40) Acc@5 100.00 (100.00)
=> Estimating diagonals of the fisher information matrix...
Estimate the fisher information of the parameters:
one
next:
Sun Mar 13 15:41:41 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.42.01 Driver Version: 470.42.01 CUDA Version: 11.4 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... On | 00000000:8A:00.0 Off | N/A |
| 33% 30C P8 29W / 350W | 1MiB / 24268MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 1 NVIDIA GeForce ... On | 00000000:8B:00.0 Off | N/A |
| 33% 30C P8 18W / 350W | 1MiB / 24268MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 2 NVIDIA GeForce ... On | 00000000:8C:00.0 Off | N/A |
| 32% 30C P8 28W / 350W | 1MiB / 24268MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 3 NVIDIA GeForce ... On | 00000000:DA:00.0 Off | N/A |
| 33% 31C P8 25W / 350W | 1MiB / 24268MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 4 NVIDIA GeForce ... On | 00000000:DB:00.0 Off | N/A |
| 33% 47C P2 134W / 350W | 3616MiB / 24268MiB | 38% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 5 NVIDIA GeForce ... On | 00000000:DC:00.0 Off | N/A |
| 33% 32C P8 20W / 350W | 3MiB / 24268MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 6 NVIDIA GeForce ... On | 00000000:DD:00.0 Off | N/A |
| 40% 29C P8 27W / 350W | 3MiB / 24268MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 7 NVIDIA GeForce ... On | 00000000:DE:00.0 Off | N/A |
| 39% 29C P8 22W / 350W | 3MiB / 24268MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 4 N/A N/A 15661 C python 3613MiB |
+-----------------------------------------------------------------------------+
Done!
Epoch: [29](test)[16/16] Time 0.012 ( 0.016) Data 0.009 ( 0.014) Acc@1 50.00 ( 52.40) Acc@5 100.00 (100.00)
TT_Task 4 finished !
Traceback (most recent call last):
File "/home/yupeng/learngene/learngene/inheritable_cifar100_wEWC.py", line 207, in <module>
main()
File "/home/yupeng/learngene/learngene/inheritable_cifar100_wEWC.py", line 203, in main
book.save(r'./imagenet_exp/outputs/i_inheritable_random_trainnum200_wEWC/i_inheritable_cifar100_wEWC.xls')
File "/home/yupeng/anaconda3/envs/learngene/lib/python3.9/site-packages/xlwt/Workbook.py", line 710, in save
doc.save(filename_or_stream, self.get_biff_data())
File "/home/yupeng/anaconda3/envs/learngene/lib/python3.9/site-packages/xlwt/CompoundDoc.py", line 262, in save
f = open(file_name_or_filelike_obj, 'w+b')
FileNotFoundError: [Errno 2] No such file or directory: './imagenet_exp/outputs/i_inheritable_random_trainnum200_wEWC/i_inheritable_cifar100_wEWC.xls'
(learngene) yupeng@ubuntu:~/learngene/learngene$
FileNotFoundError: Couldn't find any class folder in ./datasets/cifar100/base.
I wan to make continual data (source domain) and target data(target domain) on the CIFAR100 dataset. Here is my command:
first:
(learngene) yupeng@ubuntu:~/learngene/learngene$
(learngene) yupeng@ubuntu:~/learngene/learngene$ python utils/data_cifar_mk.py --num_imgs_per_cat_train 600 --path ./datasets/cifar100
Traceback (most recent call last):
File "/home/yupeng/learngene/learngene/utils/data_cifar_mk.py", line 367, in <module>
train_ll_loader, train_mm_loader, test_loader = get_cifar100_dataloaders(num_tasks, \
File "/home/yupeng/learngene/learngene/utils/data_cifar_mk.py", line 331, in get_cifar100_dataloaders
dataset_train = GenericDataset_ll(dir_name, 'cifar100','base', task_iter_item = i, subtask_class_num=subtask_classes_num, \
File "/home/yupeng/learngene/learngene/utils/data_cifar_mk.py", line 108, in __init__
self.data = datasets.ImageFolder(split_data_dir, self.transform)
File "/home/yupeng/anaconda3/envs/learngene/lib/python3.9/site-packages/torchvision/datasets/folder.py", line 310, in __init__
super(ImageFolder, self).__init__(root, loader, IMG_EXTENSIONS if is_valid_file is None else None,
File "/home/yupeng/anaconda3/envs/learngene/lib/python3.9/site-packages/torchvision/datasets/folder.py", line 145, in __init__
classes, class_to_idx = self.find_classes(self.root)
File "/home/yupeng/anaconda3/envs/learngene/lib/python3.9/site-packages/torchvision/datasets/folder.py", line 221, in find_classes
return find_classes(directory)
File "/home/yupeng/anaconda3/envs/learngene/lib/python3.9/site-packages/torchvision/datasets/folder.py", line 40, in find_classes
classes = sorted(entry.name for entry in os.scandir(directory) if entry.is_dir())
FileNotFoundError: [Errno 2] No such file or directory: './datasets/cifar100/base'
(learngene) yupeng@ubuntu:~/learngene/learngene$
second (after I created a folder named base in ./datasets/cifar100/):
(learngene) yupeng@ubuntu:~/learngene/learngene$
(learngene) yupeng@ubuntu:~/learngene/learngene$ python utils/data_cifar_mk.py --num_imgs_per_cat_train 600 --path ./datasets/cifar100
./datasets/cifar100/base
Traceback (most recent call last):
File "/home/yupeng/learngene/learngene/utils/data_cifar_mk.py", line 368, in <module>
train_ll_loader, train_mm_loader, test_loader = get_cifar100_dataloaders(num_tasks, \
File "/home/yupeng/learngene/learngene/utils/data_cifar_mk.py", line 332, in get_cifar100_dataloaders
dataset_train = GenericDataset_ll(dir_name, 'cifar100','base', task_iter_item = i, subtask_class_num=subtask_classes_num, \
File "/home/yupeng/learngene/learngene/utils/data_cifar_mk.py", line 109, in __init__
self.data = datasets.ImageFolder(split_data_dir, self.transform)
File "/home/yupeng/anaconda3/envs/learngene/lib/python3.9/site-packages/torchvision/datasets/folder.py", line 310, in __init__
super(ImageFolder, self).__init__(root, loader, IMG_EXTENSIONS if is_valid_file is None else None,
File "/home/yupeng/anaconda3/envs/learngene/lib/python3.9/site-packages/torchvision/datasets/folder.py", line 145, in __init__
classes, class_to_idx = self.find_classes(self.root)
File "/home/yupeng/anaconda3/envs/learngene/lib/python3.9/site-packages/torchvision/datasets/folder.py", line 221, in find_classes
return find_classes(directory)
File "/home/yupeng/anaconda3/envs/learngene/lib/python3.9/site-packages/torchvision/datasets/folder.py", line 42, in find_classes
raise FileNotFoundError(f"Couldn't find any class folder in {directory}.")
FileNotFoundError: Couldn't find any class folder in ./datasets/cifar100/base.
(learngene) yupeng@ubuntu:~/learngene/learngene$
The directory structure of the files is as follows:
/home/yupeng/learngene/learngene/
├── collective-model/
├── datasets/
│ └── cifar-100/
│ ├── meta
│ ├── test
│ └── train
├── utils/
└── other codes
How to create files structure for cifar100?
您好，我想请问关于cifar100的目录结构应该是怎样的，代码似乎在寻找base文件夹下的类数据？
question about data division
hello. this work is very enlightening. but for the experimental part, I have some problems. from the appendix, i find there are 4 classes left in cifar100. they are not belong to any classes you dividied. i feel confused. why you don't use them? and what's the difference between unknown classes and open-world classes?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.