fjchange / mist_vad Goto Github PK

Hi, thanks to your brilliant work.
I'm trying to test your work with my own dataset. However, I find it hard to have my dataset embedded by C3D or I3D models. Even though you've uploaded 'models' folder, its 'init.py' is empty.
I'm looking forward to your reply!

在此处（test.py117行）代码添加print的话，发现anno_type参数为空，使得normal_sxores一直为空。
def eval_SHT(model, test_dataloader): total_labels, total_scores, normal_scores = [], [], [] for frames, anno_type, _, annos in test_dataloader: # print('anno_type'.format(anno_type)) frames = frames.float().contiguous().view([-1, 3, frames.shape[-3], frames.shape[-2], frames.shape[-1]]).cuda() with torch.no_grad(): scores, feat_maps = model(frames)[:2] if args.ten_crop: scores = scores.view([-1, 10, 2]).mean(dim=-2) for clip, score, anno in zip(frames, scores, annos): score = [score.squeeze()[1].detach().cpu().item()] * args.segment_len total_scores.extend(score) total_labels.extend(anno.tolist()) if anno_type == 'Normal': normal_scores.extend(score) return eval(total_scores, total_labels, normal_scores)

Hello, please tell me that I need to call cal_false_alarm function when running test.py, and np. sum (scores * (1-np. array (labels)) is equal to 0 when running, which leads to infinity of FAR.

Hello,

I'm a 2nd year master student at University of Bucharest AI program, and I want to use your approach on my own dataset but I can't find information on how the features from I3D and C3D are produced. Could you give me more insight into how to use your method on another datasets?

Thanks.

1. The code in UCF_C3D_train.py in the stage2training folder is the same as UCF_C3D_options.py in the configs folder. I doubt you are making a mistake?

2. The description of "## 5. Stage 2 training" in Training_Guideline.md says：

% for UCF_I3D
bash UCF_I3D_train.sh

However, there is no UCF_I3D_train.sh and UCF_I3D_train.py in the stage2training folder.

Looking forward to your reply。

Thanks !

For each of the feature in 'SHT_I3D_RGB.h5' you provided, the size of the first dimension is not determined:
""
(47, 1024)
(16, 1024)
(27, 1024)
(21, 1024)
...
""
Could you tell me what this dimension means? Thanks!

File "testing/test.py", line 12, in
from visualization.Grad_CAM import GradCAM
File "/home/MIST_VAD/visualization/Grad_CAM.py", line 167, in
from UCFCrime.SpatioTemporal_Detection.CAM import visualize_CAM
ModuleNotFoundError: No module named 'UCFCrime'

title

作者您好，感谢您非常优秀的代码，
我想请教一下在参数设置中的MIL.pth文件怎么没有啊？
parser.add_argument('--model_path',type=str,default='/jiachang/Weakly_Supervised_VAD/Datasets/MIL.pth')
这个.pth文件是要在第一步中生成还是在read.me中的连接下载啊？但是我看里面没有

没有找到阶段2，UCF_C3D_train.py文件中使用的Train_TemAug_Dataset_C3D函数

For the Grad-cam, the stage-1 output feature is n*1024 and n scores after self.regressor(x), however the requirement of Grad-cam feature map is at least 3 dim, so if there is any ways to deal with the stage -1 visualization with grad-cam? Looking forward to your answers.

你好，感谢你的开源工作。
我在运行你的代码时遇到了一些问题，想咨询一下：
1.def eval_UCF(args,model,test_dataloader): 中
调用eval(total_scores, total_labels）的参数与定义的参数不一致。
将其改为eval(total_scores, total_labels, n_scores)后，经debug发现 n_scores为空列表
请问正确调用是什么样的
2.我从UCF数据集中找出了Temporal_Anomaly_Annotation_New.txt文件中所记录的视频放到了一个文件夹内。执行make_h5.py将这些视频转化为了一个H5文件，我在 OneDrive上下载你所提供的所有文件并修改了constant.py文件。
执行 python test.py --gpus 0 --MODEL UCF_C3D 得到的结果 为AUC 59.73%, GAP 10.07%
执行 python test.py --gpus 0 --MODEL UCF_I3D 得到的结果 为AUC 62.59%, GAP 7.95%
这与Tabel 的 结果相差太大了， 请问你用的UCF中哪些视频测试的结果？
3.论文里 自引导注意力编码器中 F1 和 F3 输出了的 2k 通道的特征图，k是指什么我不太明白，以及最后的输出结果只给出了异常得分，我看demo里还有异常类别。那么MIST的输出是仅输出片段级的每帧得分 还是既有得分又有异常类别呢？

Has anyone succeeded in reproducing it? If so, can you contact me. My wechat is: wjc970926. QQ is 2363846439. Thank you very much

code please ...!!!!!!

Hello, please tell me how to generate the pre-training model file in MIST, because I want to run your algorithm with my own data set.

hello:
I encountered this problem while reproducing your work ,when set num_wokers =0 ,AUC close to the accuracy of the paper.but ,set to not 0,AUC decreased by four percentage .Have you encountered this problem?
Looking forward to your reply.

Hi, thanks for the amazing work. I would like to train MIST on UCF-Crime from scratch. As I see, the training code for stage 1 is for SHT. What needs to be modified in order for it to work on UCF?

在stage1training/train_MIL_Generator.py中的，
第136行： parser.add_argument('--model_path',type=str,default='/jiachang/Weakly_Supervised_VAD/Datasets/MIL.pth')
请问里面的MIL.pth是什么文件呢？

Thanks for your awesome work and complete code. I think it is one of the most complete framework in WSVAD.
The code is of high quality, but after reading your code, I am a little confused about two issues:

1. What is the setting of feature extracting before generate PLs? Is it the same as the test phase? Do you feed the whole video as input or feed every single clip into the backbone and concat them?
2. The official link of Shanghai Tech dataset lapsed. Can you give another solid link or upload your h5-file？

There are no download option for any of these file

Hi Jiachang, regarding SHT_Frames.h5 file, did you create the file through the file make_h5.py?

最好的模型也是0.877是什么问题

冯同学您好，仓库中代码好像不全，缺少了第二阶段ucf-i3d的代码。请问您方便更新一下仓库吗？万分感谢！

Hi！
We're still trying to implement your code these days. . We have successfully obtained the H5 file of UCF_CRIME. But we are having trouble with test.py. Can you give me some help?

We hope you success in your work!

Hello, excuse me, I only have one gpu now.
model=BalancedDataParallel(int(args.batch_size2args.clip_num/len(args.gpus)*args.gpu0sz),model,dim=0,device_ids=args.gpus)
TypeError: object of type 'int' has no len()

I have downloaded your onedrive, including all the files of Baidu cloud and the Shanghai tech dataset.
Following your tutorial, I updated constant after downloading all weights. Then no new files were found in the folder.
Then I directly opened train in the first training stage, stage1 training_ MIL_ Generator.
In model_ path_ Pre and model_ Path and pseudo_ Labels_ The corresponding file cannot be found in dir.
Thank you for your excellent work.
We look forward to your reply.

tostring( ) is not supported in Python > 3.6, so I use tobytes( ) instead, but it looks like something wrong happened in frame. It has NULL so it can't work. Can you help me?

In the paper, "We uniformly sample L subsets from these video clips, and each subset contains T consecutive clips, forming L sub-bags"
In section 4.2, "Here, We adopt hyperparameters L = 32, T = 3, ".
But in "stage1training/train_MIL_Generator.py" of the released code, the "--part_len" is set to 7 as default for shanghaitech dataset.
How to set the hyperparameter T?

Thanks for your great work.
Looking forward to your reply.

When I test on the UCF-Crime dataset using python test.py --gpus 0,1,2,3 --MODEL UCF_C3D --vis_UCF,
FileNotFoundError: [Errno 2] No such file or directory: 'data/Test_Spatial_Annotation.npy'
I cannot find this file, could you provide this file?

Hello, excuse me, I run SHT.I3D_train.py to appear 'Namespace' object has no attribute 'MODEL'

kl=F.binary_cross_entropy(outputs_mean,labs,reduction='mean')
loss=args.lambda_kl*kl+loss

lambda_kl is missing in train_MIL_Generator.py, I can not find this hyperparameters in the paper too. Can you give the specific value of lambda_kl？

According to other issues, I successfully extracted the features of the ShanghaiTech dataset, but I found that it is different from the features you provided. Can you tell me the details of extracting features? It is best to provide the code. Thank you very much.

看了之前的问题：#7 (comment)
好像不应该改dataset.py吧，应该直接改test.py的load_model_dataset方法，只读取model出来，然后再改eval_UCF方法，从一个mp4文件中直接读取frame，进行预测。
但作为一个伸手党，厚着脸皮问问大神，能不能给个run prediction代码，直接读取一个视频文件，然后预测，然后输出视频结果呢？ ლ(╹◡╹ლ)

@fjchange

Hello very good paper,

However, many bugs can be reported in your directory.
If you want I can collaborate in a branch to clean up all this.
But it's very hard to debug knowing that Shanghai tech dataset is not available.
Do you plan to put the code for UCF soon (even if there are bugs)? because anyway shanghai tech contains bugs
And can you post data/test_frame_mask?

I'm trying to reproduce your results on ShanghaiTech, but it seems like I have to make SHT_Frames.h5, instead I would like to directly read from the raw videos, is there an updated version of dataset.py or has anyone tried to do so.

Hi,

First, congratulations for the great paper and thank you for publishing the code.

Unfortunately, I'm not able to reproduce the results described in the paper on the UCF-Crime dataset. As described in your testing guidelines, I did the following steps:

• I downloaded the UCF-Crime dataset and I transformed the testing set into a single h5py file (for doing that I had to slightly modify your utils/make_h5.py script, see #5). I used the default value 16 for the segment_len parameter.
• I dowloaded the pretrained weights you uploaded on OneDrive.
• I modified the paths in the configs/constant.py file to adapt to my project.
• I slightly modified your testing/test.py. Particularly, I wrapped the model object using torch.nn.DataParallel, I used the key model instead of state_dict when loading the model’s parameter dictionary, and I deleted the first occurrence of return eval(total_scores, total_labels) which was making the second occurrence unreachable. I might do a pull request if you are interested.
• I tested the model by running python test.py --gpus 0 --MODEL UCF_I3D and python test.py --gpus 0 --MODEL UCF_C3D (I slightly modified the configs/options.py file to accept the --gpus argument).

With these modifications, I managed to run the inference using python=3.8, pytorch=1.7.0+cu110, and torchvision==0.8.1+cu110 but I cannot get AUC results close to yours.

Do you have any suggestion on how to reproduce the results on the UCF-Crime testing set as described in your paper?

Many thanks in advance.

Hi,
I would like to generate pseudo labels for a dataset, but there are problems when training the MIL-based generator. Can you please provide the file "MIL.pth"? Thanks.

你好，感谢你的开源工作。我的版本：python3.6.7。

我在运行你的test.py代码时，计算 FAR 遇到了一些问题，报错如下：
../utils/eval_utils.py:40: RuntimeWarning: invalid value encountered in double_scalars
发现，是eval_utils.py中：
fp=np.sum(scores*(1-labels)) return fp/np.sum(1-labels)的问题。

尝试将这两行修改为：
fp=np.sum(scores*(1-np.array(labels))) return fp/np.sum(1-np.array(labels))
但是没有效果，依然报同样的错。

而且已将test.py中的：
far=cal_false_alarm(normal_scores,[0]*len(normal_scores))
修改为：
far=cal_false_alarm(normal_scores,np.zeros(len(normal_scores),dtype=int))
还是报同样的错

Is anybody can running the stage1 and stage2 training?