visionlearninggroup / caption-guided-saliency Goto Github PK

@ramanishka
Hi, the download of "train_val_annovation.zip" always failed.
Could you offer me the link of train_val_annotation.zip and test_annotation.zip?I couln't find them.

Thanks,
Victoria

Hi, when I ran the code for testing, it gave me some error like
""
run_s2vt.py:62: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
train_vids['video_path'] = train_vids['video_id'].map(lambda x: os.path.join(cfg.path_to_trainval_descriptors, x + "_incp_v3.npy"))
run_s2vt.py:63: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
val_vids['video_path'] = val_vids['video_id'].map(lambda x: os.path.join(cfg.path_to_trainval_descriptors, x + "_incp_v3.npy"))
{'CIDEr': 0.38908523814484475, 'Bleu_4': 0.3488993782941496, 'Bleu_3': 0.4645981761823224, 'Bleu_2': 0.6031291787236476, 'Bleu_1': 0.7600115088383854, 'ROUGE_L': 0.57838682993085411, 'METEOR': 0.26565351293713196}
2990
""

Do you know what is going on the testing code? Appreciate any kinds help.

Hello,

I wonder if you don't mind sharing your pre-trained model for those who are interested in trying your system without having to go through the training ?

Thanks,
Moustafa

Hi,

Your todo was never fulfilled. Can you post the code?

Hi,

Firstly I would like to point out a small modification in README file. For training, your instruction says to run the preprocessing script "python preprocessing.py --dataset {MSR-VTT|Flickr30k} ", while it should be runs2vt.py.

While running the training script, I face following issue in s2vt_model.py:

line 24, in init
self.lstm_cell = tf.contrib.rnn.LSTMCell(num_units = dim_hidden,
AttributeError: 'module' object has no attribute 'LSTMCell'

Thanks for your excellent work! @ramanishka
I just have a GTX 1050 Ti card. When i was training for MSR-VTT, i always got out of memory error even i decrease the batch_size=1. I'd like to how much memory does it need at least?
Thanks.

It shows like this:
(base) E:\caption-guided-saliency>python run_s2vt.py --train
D:\Program Files (x86)\Anaconda3\lib\site-packages\h5py_init_.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type.
from ._conv import register_converters as _register_converters
.\experiments\msr-vtt
run_s2vt.py:61: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
train_vids['video_path'] = train_vids['video_id'].map(lambda x: os.path.join(cfg.path_to_trainval_descriptors, x + "_incp_v3.npy"))
run_s2vt.py:62: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
val_vids['video_path'] = val_vids['video_id'].map(lambda x: os.path.join(cfg.path_to_trainval_descriptors, x + "_incp_v3.npy"))
preprocessing word counts and creating vocab based on word count threshold 1
filtered words from 23667 to 23667
2018-10-09 18:25:30.891834: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
2018-10-09 18:25:31.069295: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1405] Found device 0 with properties:
name: GeForce GTX 1050 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.43
pciBusID: 0000:01:00.0
totalMemory: 4.00GiB freeMemory: 3.30GiB
2018-10-09 18:25:31.073234: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1484] Adding visible gpu devices: 0
2018-10-09 18:25:31.456236: I tensorflow/core/common_runtime/gpu/gpu_device.cc:965] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-10-09 18:25:31.459356: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0
2018-10-09 18:25:31.461481: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] 0: N
2018-10-09 18:25:31.463833: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1097] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3686 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1050 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1)
2018-10-09 18:25:31.470165: E tensorflow/stream_executor/cuda/cuda_driver.cc:903] failed to allocate 3.60G (3865470464 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-10-09 18:25:31.473418: E tensorflow/stream_executor/cuda/cuda_driver.cc:903] failed to allocate 3.24G (3478923264 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
[0, 13020, 26040, 39060, 52080, 65100, 78120, 91140, 104160, 117180]
2018-10-09 18:25:48.659384: E tensorflow/stream_executor/cuda/cuda_driver.cc:903] failed to allocate 700.42M (734439680 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-10-09 18:25:48.663051: E tensorflow/stream_executor/cuda/cuda_driver.cc:903] failed to allocate 700.42M (734439680 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
forrtl: error (200): program aborting due to control-C event
Image PC Routine Line Source
libifcoremd.dll 00007FFE094694C4 Unknown Unknown Unknown
KERNELBASE.dll 00007FFEA36A56FD Unknown Unknown Unknown
KERNEL32.DLL 00007FFEA47F3034 Unknown Unknown Unknown
ntdll.dll 00007FFEA6501461 Unknown Unknown Unknown

Hello，
When I was ready for / TestVideo and test_videodatainfo.json, I received some errors at runtime.I tried to set the shell parameter value of the Popen function to True and run it, but I couldn't get the *.jpg file.I hope to get your help.The error appears as follows：

File "/***/preprocessing.py", line 121, in extract_features_msr_vtt extract_frames(video, dst) File "/***/caption-guided-saliency-master/preprocessing.py", line 101, in extract_frames subprocess.call(video_to_frames_command, stdout=ffmpeg_log, stderr=ffmpeg_log) File "/home/***/anaconda2/envs/python27/lib/python2.7/subprocess.py", line 168, in call return Popen(*popenargs, **kwargs).wait() File "/home/***/anaconda2/envs/python27/lib/python2.7/subprocess.py", line 390, in __init__ errread, errwrite) File "/home/***/anaconda2/envs/python27/lib/python2.7/subprocess.py", line 1024, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory

Could the visualization code use on image?

hello
I did not find the code of standard visualization for Flickr30k. Could you give me some information about it.When I was training Flickr30k dataset, there was ResourceExhaustedError, two K20 can not meet the experimental requirements?I will be grateful for your help.

Hi,

The "test_videodatainfo.json" is not available from MSR-VTT official website anymore. So, right now we cannot execute "run_s2vt.py --test" anymore. It generates an error at line 67.

Can you please provide the "test_videodatainfo.json"?

Thanks,
Peratham

excuse me, can you actually reproduce the results of the paper?
Best,