yanbeic / ccl Goto Github PK
View Code? Open in Web Editor NEWPyTorch Implementation on Paper [CVPR2021]Distilling Audio-Visual Knowledge by Compositional Contrastive Learning
License: Apache License 2.0
PyTorch Implementation on Paper [CVPR2021]Distilling Audio-Visual Knowledge by Compositional Contrastive Learning
License: Apache License 2.0
Hi,
Thank you for putting out the source code. I downloaded the data as instructed and made the json file. But running your scripts gives the following error:
label_id = class_to_idx[label]
KeyError: 'BaseballPitch'
Also, when i read the ucf51_json file, there is no BaseballPitch label. Could you perhaps tell what might be the issue.
Thanks,
Hi,
When I tried to run scripts/run_ccl_ai.sh after generating the json file. I received the error below..
Traceback (most recent call last): │
File "main.py", line 569, in │
main_worker(-1, opt) │
File "main.py", line 522, in main_worker │
tb_writer=tb_writer, distributed=opt.distributed, image_size=image_size, loss_weight=opt.loss_weigh│
t) │
File "/media/data/huwang/experiments/multimodal/CCL/training.py", line 402, in train_ai_epoch │
features_img = image_model(images) │
File "/media/data/huwang/program_files/venv/CCL/lib/python3.6/site-packages/torch/nn/modules/module.p│
y", line 493, in call │
result = self.forward(*input, **kwargs) │
File "/media/data/huwang/program_files/venv/CCL/lib/python3.6/site-packages/torch/nn/parallel/data_pa│
rallel.py", line 152, in forward │
outputs = self.parallel_apply(replicas, inputs, kwargs) │
File "/media/data/huwang/program_files/venv/CCL/lib/python3.6/site-packages/torch/nn/parallel/data_pa│
rallel.py", line 162, in parallel_apply │
return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)]) │
File "/media/data/huwang/program_files/venv/CCL/lib/python3.6/site-packages/torch/nn/parallel/paralle│
l_apply.py", line 83, in parallel_apply │
raise output │
File "/media/data/huwang/program_files/venv/CCL/lib/python3.6/site-packages/torch/nn/parallel/paralle│
l_apply.py", line 59, in _worker │
output = module(*input, **kwargs) │
File "/media/data/huwang/program_files/venv/CCL/lib/python3.6/site-packages/torch/nn/modules/module.p│
y", line 493, in call │
result = self.forward(*input, **kwargs) │
File "/media/data/huwang/program_files/venv/CCL/lib/python3.6/site-packages/torch/nn/modules/containe│
r.py", line 92, in forward │
input = module(input) │
File "/media/data/huwang/program_files/venv/CCL/lib/python3.6/site-packages/torch/nn/modules/module.p│
y", line 493, in call │
result = self.forward(*input, **kwargs) │
File "/media/data/huwang/program_files/venv/CCL/lib/python3.6/site-packages/torch/nn/modules/pooling.│
py", line 563, in forward │
self.padding, self.ceil_mode, self.count_include_pad) │
RuntimeError: Given input size: (512x4x4). Calculated output size: (512x-2x-2). Output size is too smal│
l at /pytorch/aten/src/THCUNN/generic/SpatialAveragePooling.cu:47
Can you offer some help.. Thank you
Thanks a lot for your insightful work. Since I want to reproduce your method, I wonder how many computation resources you used and how long you spent training your network? Looking forward to your reply.
Hi,
Thank you for sharing such good codes and work.
When I tried to generate the json files, ../data/UCF101/UCF101_hdf5/HandstandPushups/v_HandStandPushups_g08_c01.hdf5 to ../data/UCF101/UCF101_hdf5/HandstandPushups/v_HandStandPushups_g07_c04.hdf5. It returns Cannot find the file...
I tried to regenerate the hdf5 of HandstandPushups, the problem is still there.
Really appreciate if you can offer some help!
Best,
Hu
I tried CCL baseline, audio & AI but they all have this issue
random crop
/scratch1/zhe031/envs/minic220426/lib/python3.9/site-packages/torchvision/transforms/transforms.py:890: UserWarning: Argument interpolation should be of type InterpolationMode instead of int. Please, use InterpolationMode enum.
warnings.warn(
Traceback (most recent call last):
File "/scratch1/zhe031/CCL/main.py", line 569, in
main_worker(-1, opt)
File "/scratch1/zhe031/CCL/main.py", line 463, in main_worker
get_train_utils(opt, model_parameters=parameters, av_parameters=aud_para, iv_parameters=img_para)
File "/scratch1/zhe031/CCL/main.py", line 189, in get_train_utils
train_loader = torch.utils.data.DataLoader(train_data,
File "/scratch1/zhe031/envs/minic220426/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 277, in init
sampler = RandomSampler(dataset, generator=generator) # type: ignore[arg-type]
File "/scratch1/zhe031/envs/minic220426/lib/python3.9/site-packages/torch/utils/data/sampler.py", line 97, in init
raise ValueError("num_samples should be a positive integer "
**_
ValueError: num_samples should be a positive integer value, but got num_samples=0
_**
I guess normally it is a dataloader issue. I have double-checked that my hdf5 data & audio data is not 0K (exactly have data), and the file path is the same as given in readme file. So have you suffered the same bug previously? Or do you have any idea about how to fix it?
Thanks
I run the scripts you provided and I can not get the results on UCF51 you list in your paper. Could you provide some pretrained models on each dataset?
As we can see, there are four functions for train. I found a problem, In train_epoch function which is baseline, write_to_epoch_logger() are excuted after finishing the batch calculation("for" loop). But in the other 3 training functions(train_a_epoch, train_i_epoch, train_ai_epoch), that write_to_epoch_logger() function is inside the for loop which is different from the baseline. This should be placed outside the for loop.
Meanwhile that write_to_epoch_logger() function's write log argument should be avg, not val. If you use val, it means that you are using the acc of the last input in this batch, not the average acc of the whole batch。
If my understanding is wrong, please correct me! I would be grateful if you could reply as soon as possible.
Hi,
Would it be possible to know a rough estimate of time before you release the code here?
Thank you,
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.