gsig / charades-algorithms Goto Github PK

Hi gsig, thanks for sharing this project.
I just ran into this error when executing python main.py in the pytorch directory.

Traceback (most recent call last):
File "main.py", line 59, in
main()
File "main.py", line 49, in main
top1,top5 = trainer.train(train_loader, model, criterion, optimizer, epoch, opt)
File "/net/10.79.12.41/vision/u/xw1/charades-algorithms/pytorch/train.py", line 80, in train
for i, (input, target, meta) in enumerate(part(loader)):
File "/net/10.79.12.41/vision/u/xw1/charades-algorithms/pytorch/train.py", line 77, in part
return itertools.islice(x, int(len(x)*args.train_size))
File "/vision/u/xw1/anaconda3/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 417, in iter
return DataLoaderIter(self)
File "/vision/u/xw1/anaconda3/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 242, in init
self._put_indices()
File "/vision/u/xw1/anaconda3/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 290, in _put_indices
indices = next(self.sample_iter, None)
File "/vision/u/xw1/anaconda3/lib/python3.6/site-packages/torch/utils/data/sampler.py", line 119, in iter
for idx in self.sampler:
File "/vision/u/xw1/anaconda3/lib/python3.6/site-packages/torch/utils/data/sampler.py", line 50, in iter
return iter(torch.randperm(len(self.data_source)).long())

I see that this is caused by dataloader providing 0 training data.
My opts.py is set up as the followings:

--data : "{Path}/Charades_v1_features_flow"
--dataset : "charadesflow"
--train-file and --val-file are also set to the correct csv files

Is there anything I'm missing in opts.py or any thoughts on how I could solve this error? Thanks a lot!

I followed the instructions and proceeded towards retraining the rgbnet with default parameters, and ran it for 6 epochs with other parameters unchanged, I checked the epoch#.txt files and found out that the the network is predicting the same values for each video, the values change between epochs but stay the same for each video inside the file.

As a debug step i also predicted some images using RESNET inside and it was also giving same probabilities for any picture i feed in it, I tried every architecture

Ihave attached screenshot of one epoch file and .txt files for all epochs

Uploading epoch1.txt…
Uploading epoch2.txt…
Uploading epoch3.txt…
Uploading epoch4.txt…
Uploading epoch5.txt…

As I run the dataset, I find that some videos' annotations about end time of action is bigger than the total length of that video. For example,\

name action label total length
42K3H c006 0.00 3.60;c008 0.00 5.00;c112 0.00 3.90 3.5

This c008's end time is 5s > 3.5s.
Should I just turn this 5 into 3.5 ?

Could you please tell me which algorithms results in 'test_submission_localize.txt' come from?

hI,
The Pytorch model on dropbox: https://www.dropbox.com/s/p457h2ifi6v1qdz/twostream_rgb.pth.tar?dl=1, seems being corrupted:
I have tried to unzip the file on ubuntu with: tar xvf twostream_rgb.pth.tar:

tar: This does not look like a tar archive
tar: Skipping to next header
tar: Exiting with failure status due to previous errors

Could you help to fix this problem?

Hi,
Is there any chance you have the equivalent of datasets/charadesrgb.py but in Tensorow? It would help me to process also the FC7 features instead of the RGB images.

Hi Gunnar,

Thanks for sharing the amazing codebase! I'm wondering under the classification evaluation mode (or localization), should the action score for each video / frame be the score after passed through softmax or before? From the example submission file it seems to be before (cuz there're negative values), but from the validate_video() function seems the after-softmax scores are added to the submission file.

Thanks in advance!

Boxiao

TODO. In the meantime, see the Torch codebase.

https://github.com/wykang/Charades/blob/adc58b7cfe2567f17cc7b62caf4ff4a13a1e8f22/utils/map.py#L26
m_ap = np.mean(m_aps)
m_ap = np.nanmean(m_aps)
Are these two evaluation indicators the same?

TODO

I'm using the Charades dataset class in charadesrgb.py to construct a Dataset object and then feeding it into a Dataloader. However, it seems like the mini-batches returned by my Dataloader are of dimension BxCxHxW (missing the temporal T dimension).

Is this the expected behavior? Or am I using this incorrectly?

Hi,

I came across your data set and repo while researching about video analysis.
Thank you very much for sharing your rich data set and code.

I'm using pytorch implementation and able to load the rgb model without any problem.
But I', facing problems while loading flow model downloaded from https://www.dropbox.com/s/m1hkeiwjtndt26z/twostream_flow.pth?dl=1.

Specifically, I face error @ # replace last layer block of code in __init__.py file in the final else case.
System says "Dropout does not have field in_features..."

Could you please give me some direction on how to overcome this issue?
Thanks in advance!

When optnet is turned off, the flownet does not seem to converge properly. Optnet does shrink the number of actual parameters, so some adjustment to the learning parameters might be needed. Writing this issue for reference.

Average time to load a batch of 64 optical flow stacks takes 0.054 sec on average on a SSD for training, which is all good. On a HDD it is slow for training, but testing pretrained optical flow models seems fine. Maybe installing accimage fixes this.

TODO. In the meantime, see the Torch codebase.

How can i convert the 2 dimensional (u,v) obtained from dense optical flow algorithm to the id-0000.x and id-0000.y files, help appreciated.
How is the sequence numbered like the optical flow between images 1 and 2 is labelled 1.x and 1.y ?

In the annotation file, more than one kind of action happens in one frame. For example, in sequence 46GP8, c092 and c147 are overlapping between 11.9s and 12.6s. So frames between this time period seem to have multiple labels.

46GP8,HR43,Kitchen,6,7,Yes,A person cooking on a stove while watching something out a window.,food;stove;window,A person cooks food on a stove before looking out of a window.,c092 11.90 21.20;c147 0.00 12.60,24.83

But the data loader you implemented seems to ignore it? Does it mean some frames will be appended to image_paths for multiple times?

CrossEntropyLoss will be confused as some frames have 2+ different labels?