I wanted to load the pretrained weights but on specifying --pretrained_dir as the absolute path of the weights file: soyloc.bin, I get the following error:

WARNING - main - Process rank: -1, device: cuda, n_gpu: 1, distributed training: False, 16-bits training: False
Traceback (most recent call last):
File "/FFVT-main/train.py", line 416, in
main()
File "/home/code/classifiers/FFVT-main/train.py", line 409, in main
args, model = setup(args)
File "/FFVT-main/train.py", line 93, in setup
model.load_from(np.load(args.pretrained_dir, allow_pickle=True))
File "/FFVT-main/models/modeling.py", line 379, in load_from
self.transformer.embeddings.patch_embeddings.weight.copy_(np2th(weights["embedding/kernel"], conv=True))
TypeError: 'int' object is not subscriptable

And when I try to load weights I generated(--pretrained_dir= my_ckpt.bin) after training with a custom dataset I get:

Traceback (most recent call last):
File "/FFVT-main/train.py", line 416, in
main()
File "/FFVT-main/train.py", line 409, in main
args, model = setup(args)
File "/FFVT-main/train.py", line 93, in setup
model.load_from(np.load(args.pretrained_dir, allow_pickle=True))
File "/FFVT-main/models/modeling.py", line 379, in load_from
self.transformer.embeddings.patch_embeddings.weight.copy_(np2th(weights["embedding/kernel"], conv=True))
File "/PyEnvs/base/lib/python3.8/site-packages/numpy/lib/npyio.py", line 260, in getitem
raise KeyError("%s is not a file in the archive" % key)
KeyError: 'embedding/kernel is not a file in the archive'

The vit-b_16

• only pre-trained on imagenet21k or(393mb)
• pre-trained on imagenet21k and fine-tuned on imagenet2012(331mb)

Thank you for this awesome work.

I'm getting a problem when using the training script. The problem is related with apex, can you include a command how to install it properly (I tried to do it with pip but it doesn't work)

Have a nice day

1. Hi, when I saw your code, I found that only the first 11 layers of the transformer are loaded (when feature_fusion == True)
   And the "ff_last_layer" and "ff_encoder_norm" are trained from scratch, am I right?
2. If so, what is the performance when loading 12th layer weights to ff_last_layer and norm to off_encoder_norm?
   Thanks

Hi, Thanks for your excellent work, i have a question, how many steps did you train on each dataset? and dose the batchsize you mentioned in your paper refer to a single GPU?
Thank you!