efficient-video-recognition's People
Forkers
vieozhu babajideowoyele jihwaneom tomchen-ctj carpedkm talalwasim jeykigung allanchan339 yixing-li zylwithxy sanyam83 taoyang1122 jong980812 patrick-john-ramos whuhxbefficient-video-recognition's Issues
Could you share k600/k700 performance? (EOM)
Ask for finetuning scripts
Hi, could you please provide training scrips for finetuning ? I want to reproduce the results of finetuning reported in your paper.
Thanks!
when the checkpoints and scripts are is open for us?
Great works! when the checkpoints and scripts are is open for us? Thanks very much!
Missing running scripts
Great works! But where is the 'scripts ' floder?
Visualization of cross-frame attention maps
Hi,
Congratulations on your awesome work. If possible, could you also share the script for visualizing cross-frame attention maps (fig. 6 in the paper)?
Thanks in advance.
train error
[17:29:39.025738] Not resuming from a checkpoint.
Traceback (most recent call last):
File "main.py", line 238, in
main()
File "main.py", line 148, in main
for i, (data, labels) in enumerate(train_loader, resume_step):
File "/root/miniconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 521, in next
data = self._next_data()
File "/root/miniconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data
return self._process_data(data)
File "/root/miniconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data
data.reraise()
File "/root/miniconda3/lib/python3.8/site-packages/torch/_utils.py", line 425, in reraise
raise self.exc_type(msg)
File "av/error.pyx", line 78, in av.error.FFmpegError.init
TypeError: init() takes at least 3 positional arguments (2 given)
About params config
Hi, thanks for your good work!
My GPUs are limited, only 4 2080Ti.
I set the batchsize=48, lr = 1e-4, backbone is ViT-L-14. Can I get an acceptable result?
Could you give me some suggestions about the params config?
How to use the CLIP backbone with other dataset?
I am trying to use the CLIP backbone with the UCF crime dataset. How should I proceed with that?
Is it possible that the Frozen CLIP model saw the validation set of Kinetics
Hi,
Congratulations on your awesome work. It is interesting and meaningful.
I have a little concern about this work. I was wondering if it is possible that the dataset on which the CLIP is pre-trained contains part of Kinetics validation video/images.
Thanks in advance.
accuracy of vitB16 and vitL_14
I'm very interested in your work and reproduced it. I used the scripts statements you gave , but the reproduction results on the k400 dataset are a bit lower than the results in the paper . I got the accuracy: vitB16_8f:78.87% (82.17%),vitL14_8f:84.76% (86.3%) . I used 4 rtx3090s , the environment is also configured according to the statement in the github. Can you give me some suggestions?
Enquiry on the accessibility of SSV2 configs and checkpoints
Hi, I just wonder if the configs and checkpoints for SSV2 will be available in the future.
Saved checkpoints corrupted?
I used your code as it is to train a VitB16 model. But when I try to load the saved checkpoints, they all give the same error:
"PytorchStreamReader failed reading zip archive: invalid header or archive is corrupted"
Tried to inspect the checkpoint file using pickletools, but got the following error in each case.
"UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 63: ordinal not in range(128)"
Is this something you encountered as well?
Does action recognition not require the start time and end time?
Hi, I have a question as a beginner for action recognition task.
My question is same with title, does action recognition not use start time(frame) and end time information?
I confirmed kinetics400 dataset has time information that contains when the action starts and ends.
If there is many action in one video file, I think this time information is needed, but in this repo, I understand EVL only requires the label of video, not include time information.
Could you tell me about this?
Thank you!
Using ViT-L-14-336px checkpoint
First of all, congrats on the publication of this paper. And thanks for sharing the code.
I tried using the ViT-L-14-336px model, as you got the best result with this one. However, I have seen that the backbone for this model is not defined:
efficient-video-recognition/vision_transformer.py
Lines 307 to 326 in b282e8b
Can you add it? I tried it on my own, but not successfully.
Thanks in advance.
bug
Hello!When I run this code, it shows:
AttributeError: 'dict' object has no attribute 'visual'
Traceback (most recent call last):
File "main.py", line 229, in
if name == 'main': main()
File "main.py", line 114, in main
num_frames=args.num_frames,
File "/home/wyl/efficient-video-recognition-master/model.py", line 170, in init
backbone_config = self._create_backbone(backbone_name, backbone_type, backbone_path, backbone_mode)
File "/home/wyl/efficient-video-recognition-master/model.py", line 202, in _create_backbone
state_dict = weight_loader_fn(backbone_path)
File "/home/wyl/efficient-video-recognition-master/weight_loaders.py", line 13, in load_weights_clip
clip_model = clip_model.visual
Do you konw why?
ssv2 data sampling
Thanks for your work. But when I experimented on SSv2 dataset, as described in the paper that using the TSN style to sample frames, the sampling rate is set to -1, which will perform wrong sampling result in '_generate_temporal_crops'.
Could you kindly provide the ssv2 runable scripts?
Wprev and Wnext in the formula of the cross-attention part of the paper don't seem to make sense
Hello, how should I understand Yatten in the formula of the cross-attention part of the paper, where Aprev and Anext is already a learnable tensor used to capture optical flow, and Wprev and Wnext is a re-learned feature, which will wouldn't make sense in my opinion
any plans for adding more comments?
Thanks a lot for open-sourcing this project!
I would like to mention that adding more comments to the codebase could help a lot of people to understand what is going on!
batchsize vs training iterations
I had a couple of very basic questions.
Since you define the training length by number of iterations rather than number of epochs, does that mean increasing the batch size would require decreasing iterations accordingly to stay consistent with the 50 epoch training you report?
Because, from what I understood, 50k iterations at a batchsize of 256 would approximately mean 50 epochs (around 950 iterations/epoch for k400). Am I understanding this correctly or have I missed something?
Also, the number of workers defined is the num of workers per GPU or is it total workers across all?
Thank you in advance.
。
fix DDP mode saving checkpoint
in the main.py file 193 line should be changed to
if (i + 1) % args.save_freq == 0 and dist.get_rank() == 0:
only master node save checkpoint saving function
Error when running with finetune backbone
I am getting a strange error when trying to use the finetune backbone option.
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [128, 197, 768]], which is output 0 of torch::autograd::CopySlices, is at version 3; expected version 0 instead.
Do you have any idea why this might be happening?
Talal
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.