Hello, I try to use download_videos.py download the dataset.The code run but the videos aren't in the folder.

Is there anything wrong in the code?

Hello, I try to train the network. My tensorflow version is 1.13.2. I follow your tutorial and run sh run_training.sh.

I found that the IR_setting don't work fine.The mission has no computation(great57) for hours and doesn't show errors.

I found that the code in framework.py line 185 :train_res = self.sess.run(self.train_targets) don't work and return no response.

Can you help me solve the bug? I'm not quite familiar with tensorflow.@chengxuz

Can't find any codes for TwoPathway models, Are there any clues about them available?

Hi everyone, first thanks a lot for your outstanding Paper and open resource code!
I am trying to train some videos without annotated, and get the video level representation, e.g., get a embedding likes BERT doing (I am a newer in CV). When after reading your code, I am confused in this line

Hello! I have a question on the training batch size.
I train the 3dresnet18 and I print the tensor shape.

The output tensor shape is (64,256,2,7,7) and(160,256,2,7,7)
What's the meaning of batch size 160? I didn't find this parameter.

I've tried to load the checkpoint linked in the README using run_finetune_HMDB.sh as a template. After patching up some errors and hardcoding the checkpoint location in framework.py, I get this error during the restore process:

Restore from /content/video_pub_cache/models/vd_finetune/HMDB/vd_3dresnet_sc/model.ckpt
2021-05-22 05:01:02.960066: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at save_restore_v2_ops.cc:184 : Not found: Key instance/transfer_dense/bias not found in checkpoint
Traceback (most recent call last):
  File "/tensorflow-1.15.2/python3.7/tensorflow_core/python/client/session.py", line 1365, in _do_call
    return fn(*args)
  File "/tensorflow-1.15.2/python3.7/tensorflow_core/python/client/session.py", line 1350, in _run_fn
    target_list, run_metadata)
  File "/tensorflow-1.15.2/python3.7/tensorflow_core/python/client/session.py", line 1443, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.NotFoundError: 2 root error(s) found.
  (0) Not found: Key instance/transfer_dense/bias not found in checkpoint
	 [[{{node save/RestoreV2}}]]
  (1) Not found: Key instance/transfer_dense/bias not found in checkpoint
	 [[{{node save/RestoreV2}}]]
	 [[save/RestoreV2/_133]]
0 successful operations.
0 derived errors ignored.

Could you show a minimal example of creating a model of the correct architecture and loading the pretrained checkpoint?

Hi, I have another two questions to ask you. First, just as this line

hello, when i start training directly, i run the run_training.sh, but it shows there is no model.ckpt-50000 file. Can you help me solve this problem? Thanks!

Hello! I can use Jupyter Notebook with your help of installing MongoDB.

I found that there are two lines of 3D resnet.

What's the meaning of"re-"?

hello dear author, what is your Tensorflow version?

When I try to run

python3 download_videos.py --csv_path kinetics400/train.csv --save_dir data --save_raw_dir raw_data --len_idx 246535

I got error

  0%|                                                                       | 13/219782 [00:15<73:01:04,  1.20s/it]
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib/python3.7/multiprocessing/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "download_videos.py", line 118, in download_one_video
    video = yt.streams.filter(
  File "/home/hongbo_miao/.local/lib/python3.7/site-packages/pytube/__main__.py", line 307, in streams
    self.check_availability()
  File "/home/hongbo_miao/.local/lib/python3.7/site-packages/pytube/__main__.py", line 267, in check_availability
    raise exceptions.VideoPrivate(video_id=self.video_id)
pytube.exceptions.VideoPrivate: --bO6XwZ9HI is a private video
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "download_videos.py", line 190, in <module>
    main()
  File "download_videos.py", line 186, in main
    multi_process_download(args, csv_data, curr_len)
  File "download_videos.py", line 173, in multi_process_download
    total=curr_len))
  File "/usr/local/lib/python3.7/dist-packages/tqdm/std.py", line 1178, in __iter__
    for obj in iterable:
  File "/usr/lib/python3.7/multiprocessing/pool.py", line 748, in next
    raise value
pytube.exceptions.VideoPrivate: --bO6XwZ9HI is a private video is a private video

Is this error related with pytube version? I am using pytube 10.7.2.
May I know which pytube version you are using? Thanks

Hello I got some result on the dataset.

topn: {'top1_10NN': 0.080424786}
I know it's not the accuracy. I read your paper,but don't know the meaning of Nearest neighbor validation performances.

What does it mean?

Hello,I want to train the model with multi_gpu, I change the code in run_training.sh"gpu=0" to "gpu=0,1"

the code in train_vie.py got an error.
assert len(nn_clusterings) == multi_gpu.
because len(nn_clusterings) is 1 but mulit_gpu is 2.

Is there anyother code I need to change?

Hello! It's me again. Thanks for your help,I can start training in my dataset. (The IR result accuracy is only 10%. ) Whatever, I continue training in with LA. However , it comes an error.

I didn't find its difinition. Is it an easy-solving bug?

Hi, UCF101 contains train splits from 1 to 3 and test splits from 1 to 3, which train split and test split did you use to train and evaluate transfer model?

Hi Chengxu,thank you very much for your open source and paper,that's very helpful .
But I encounter somes questions about the code in download_videos.py.
There are just five items in the file named train.csv : label,youtube_id,time_start,time_end,split.
However,the code named download_vedios.py needs the sixth items of the file .
look at the code
'id': line_split[1], 'sta': int(float(line_split[2])), 'end': int(float(line_split[3])), 'train': line_split[4], 'flag': int(line_split[5]), 'indx': curr_indx}
So,what's the function of the flag?Is there has some error in the code or I've made some mistakes about the code.
Look forward to your reply

Hi, have you ever tried training resnet3d model with only IR loss. Now I am implementing the pytorch version, however I find that after training for 250000 steps with IR loss, I can only reach top1_10nn about 20%. Is it necessary to use LA loss in order to get the best performance?

Hi, when you trained resnet3d model, you first pretrained it with IR loss, and after 50, 000 steps you trained it with LA loss. I want to ask whether you load the labels from IR to initialize the clustered_labels in training LA?

Hey, great work! Do you plan to release a PyTorch version?

Best.

For kinetics 400, which split did you use to get the metrics in your paper? validate or test?

can't tell all these setting functions in vd_single_frame_fx.py corresponding to which experiment in your paper. If I want to train a VIE-Single, which setting should I choose?

Hello! I meet a new question.
the paper's training dataset is kinetic dataset which contains 400 labels and about 240000 videos. I found that the parameter of kmeans_k is 10000 in your training strategy.
I want to train the model in jester dataset which contains 27 labels and about 120000 videos. Which is smaller and simpler than kinetic dataset.
I train VIE-3DResNet-IR training as pretraining for the VIE-3DResNet for 10 epoch following the tutorial.The top NN result is 10% .However the VIE-3DResNet's result is not good .No matter I change the kmeans_k value. The result can't reach 10% and the accuracy decrease .
However if I continue training the IR model, The top NN result can reach 60%.

Different datasets have different training parameters.I use the baseline loss function but get the better result which confuse me. Can you provide some advice for me or is there more parameter I forget to change?

Hi chengxu,

Thanks a lot for your excellent work and open resource code! Here is my problem.
When I want to download kinetics like your version, it seems that you did not provide the kinetics_train.csv.

Kinetics-400 contains two folders "train" and "val" in my mind which exsits no kinetics_train.csv like you said. How should I get it?

Looking forward for your reply and thanks!

Hi, now I am training the slowfast model, and I will have to spend about 370ms per step with batch size 64 and I am training on 4 Nvidia V100 cards. Is this speed normal and any suggestions to accelerate the training process?

Thank you

Hi, Now I can train the model sucessfully and thanks for your great work. Now I met a small question. I want to train the model in another dataset. I copy the code and rename the filename such as VIE-master2. And I want to load the original model .
Maybe some code needed to be modified, the terminal shows **Couldn't match files for checkpoint ** in the VIE-master with the ckpt I need to load. I change the cache_dir to VIE-master2 in run_training.sh.But why the code read the VIE-mater filefolder. And I didn't find any parameter of VIE-master.
Can you help me solve this little question?