gjy3035 / c-3-framework Goto Github PK

我看代码生成的GT的人数求和 和给的标注的人数不一样，是不是应该乘个系数，使得密度图求和就等于标注的人数？？

验证集与测试集。您好，我看您的代码中，验证集即为测试集，请问验证集和测试集不用区分吗？还是因为图片数量少，所以可以用测试集当作验证集处理。

How do you realize the test process of SANet? Can you provider the code for patch-based test？ Thanks!

How to add pixel values of the density map. If there is code available?

Is there any code to process UCSD dataset? Thanks for your help!

你好，谢谢你的代码，但是我在运行的过程中，出现
RuntimeError: DataLoader worker (pid(s) 11208, 3676, 5316, 10956, 9808, 11116, 1212, 9512) exited unexpectedly
这样的错误，请问有人能解决吗？

It is the test program.
**when the program is run this row of code： f1 = plt.figure(1)
it stops

describe the issue and paste the code segment which is in the mathplot lib firstly

 figManager = new_figure_manager(num, figsize=figsize,
                                        dpi=dpi,
                                        facecolor=facecolor,
                                        edgecolor=edgecolor,
                                        frameon=frameon,
                                        FigureClass=FigureClass,
                                        **kwargs)#stop here

        if figLabel:
            figManager.set_window_title(figLabel)
            figManager.canvas.figure.set_label(figLabel)

When it reaches the "**kwargs)" row, the program stops

and the error code of running the program is
Process finished with exit code -1073740791 (0xC0000409)

How to fix the issue？ @gjy3035

请问：1.LOG_PARA参数指什么，为什么那么每张图片求和后要除以它？2.可以简单描述一下标签归一化作用么！？谢谢！

小白不知道GCC 数据集的cc cl rd是什么意思呀，代表什么呢

请问这个怎么解决

Are there available pre-trained models for the whole network, for testing without the need for any training?

Thanks

If we run the test.py code on a single image it returns different density map each time(stored in pred_map) for CSRNet.

I conducted the experiment on CSRNet_UCF50 model, the res is : mae:196.5/mse:235.3, which is not match with mae:266.1/mse:397.5 in https://github.com/gjy3035/Awesome-Crowd-Counting#SPN.

validate_V2 in trainer.py line202 has referenced a undefined value mse, and it could cause run error

self.train_record = update_model(self.net,self.optimizer,self.scheduler,self.epoch,self.i_tb,self.exp_path,self.exp_name, \
            [mae, **mse**, loss],self.train_record,self.log_txt)

第2.1节中，

More processing details are displayed in Table 2.

应该是Table 1吧

另外有些不太清楚的地方。生成密度图中的kennel size 是什么意思。是生成密度图时，高斯核函数的参数吗？还是说一个人头在原图分辨率中表示为一个15*15的大小？

hi!First of all ,thanks for your subtle framework!!but i have some questions,

1. in code,for example in datasets/SHHA/pre_data.py,you resize the imgs' size so that they can be divided by 16 and save to the dstRootAttr,but after i download the processed data you provided in readme,i found that the imgs' size are not divisible by 16?

2）what is these code for？

dst_wd= wd/16*16
rate_wd  = float(dst_wd)/float(wd)
dst_ht = ht/16*16
rate_ht = float(dst_ht)/float(ht)
gt_x = (gt[:,0]*rate_wd).astype(np.int64)
gt_y = (gt[:,1]*rate_ht).astype(np.int64)`

why should gt_* multiply rate_*?
thanks a lot!!

@SelinaFelton

对于CMTL的复现，代码我已经检查完成。很遗憾，在尝试了多组参数后，网络依然无法收敛。

之前我也遇到过pytorch0.2的回归网络在0.3及其以上版本中无法收敛的问题。你可以尝试使用CMTL的初始化代码，对网络进行初始化操作。

也可以暂时分类loss权重置零，仅仅训练回归网络。如果有问题，可以继续在这里讨论。

您好, 因為大學畢業專題要做這方面的內容, 所以想先執行一次大概看一下整體的效果, 目前我做到train完的步驟, 然後就有點沒有頭緒要如何更改了QQ 想請您大概給我一點方向, 是要調整test.py裡面的內容嗎？ 有點不太清楚如何去做更改 希望您能指點一下 謝謝！

I want to auto count material number using your framework,could you please help to evaluate its feasibility,the quantity must be accurate,not a rough estimate,thanks in advance for your reply!

To run this framework on windows, I need to install pytorch, but when I instalingl pytorch with "conda install pytorch", my python version was updated from 2.7 to 3.6, so I couldn't run the initial code.
How to fix this problem

When I ues CSRNet to train with single GPU, I meet the error of this :
self.class.name, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for CrowdCounter:
Missing key(s) in state_dict: "CCN.frontend.0.bias", "CCN.frontend.0.weight", "CCN.frontend.2.bias", "CCN.frontend.2.weight", "CCN.frontend.5.bias", "CCN.frontend.5.weight", "CCN.frontend.7.bias", "CCN.frontend.7.weight", "CCN.frontend.10.bias", "CCN.frontend.10.weight", "CCN.frontend.12.bias", "CCN.frontend.12.weight", "CCN.frontend.14.bias", "CCN.frontend.14.weight", "CCN.frontend.17.bias", "CCN.frontend.17.weight", "CCN.frontend.19.bias", "CCN.frontend.19.weight", "CCN.frontend.21.bias", "CCN.frontend.21.weight", "CCN.backend.0.bias", "CCN.backend.0.weight", "CCN.backend.2.bias", "CCN.backend.2.weight", "CCN.backend.4.bias", "CCN.backend.4.weight", "CCN.backend.6.bias", "CCN.backend.6.weight", "CCN.backend.8.bias", "CCN.backend.8.weight", "CCN.backend.10.bias", "CCN.backend.10.weight", "CCN.output_layer.bias", "CCN.output_layer.weight".
Unexpected key(s) in state_dict: "er", "ecord", "e", "", "h".

How to fix this? thank you

你好，我想问下关于各个数据集生成ground truth的问题，我看了下你的代码，除了SHA外，其他的数据集生成方式相同，都是用的sigma=4的高斯核，这个sigma值的选取有技巧吗，为什么都是4，在CSRNet论文中处理SHB用的是sigma=15，这个没有影响吗？

Can I continue training from some epoch(like epoch 100 )? It seems like the framework doesn't support this function? Thanks.

你好，谢谢你的代码，但是我在运行的过程中，出现
RuntimeError: DataLoader worker (pid(s) 11208, 3676, 5316, 10956, 9808, 11116, 1212, 9512) exited unexpectedly
这样的错误，请问有人能解决吗？

How many epochs to get the corresponding results of the dataset and model? The default setting in config.py is 100 and there is no explanation. And will you release the trained models?

Could you tell me how to use your framework to train the model in Mall or UCSD?

RuntimeError: Error(s) in loading state_dict for CrowdCounter:
Missing key(s) in state_dict: "CCN.de_pred.0.conv.bias", "CCN.de_pred.0.conv.weight", "CCN.de_pred.1.conv.bias", "CCN.de_pred.1.conv.weight", "CCN.frontend.0.weight", "CCN.frontend.1.running_var", "CCN.frontend.1.bias", "CCN.frontend.1.weight", "CCN.frontend.1.running_mean", "CCN.frontend.4.0.conv1.weight", "CCN.frontend.4.0.bn1.running_var", "CCN.frontend.4.0.bn1.bias", "CCN.frontend.4.0.bn1.weight", "CCN.frontend.4.0.bn1.running_mean", "CCN.frontend.4.0.conv2.weight", "CCN.frontend.4.0.bn2.running_var", "CCN.frontend.4.0.bn2.bias", "CCN.frontend.4.0.bn2.weight", "CCN.frontend.4.0.bn2.running_mean", "CCN.frontend.4.0.conv3.weight", "CCN.frontend.4.0.bn3.running_var", "CCN.frontend.4.0.bn3.bias", "CCN.frontend.4.0.bn3.weight", "CCN.frontend.4.0.bn3.running_mean", "CCN.frontend.4.0.downsample.0.weight", "CCN.frontend.4.0.downsample.1.running_var", "CCN.frontend.4.0.downsample.1.bias", "CCN.frontend.4.0.downsample.1.weight", "CCN.frontend.4.0.downsample.1.running_mean", "CCN.frontend.4.1.conv1.weight", "CCN.frontend.4.1.bn1.running_var", "CCN.frontend.4.1.bn1.bias", "CCN.frontend.4.1.bn1.weight", "CCN.frontend.4.1.bn1.running_mean",

Hello！I tried to use CSRNet to reproduce on SHHA, but only get MAE/MSE 76.3/118.52. So I have some question about SHHA setting.
(I use batch=8, cropped images(576,768), lr=1e-5, epoch=150)

1. Train on SHHA use one batch or multi batch？I find it support both one and multi batch in the code.
2. When training, use the original image or the cropped image(576,768) ?
3. About learning rate and epoch setting?(same as SHHB? lr=1e-5, epoch =100?)
4. When testing, use the original image‘s size may not multiple of 16, how to deal with it？

Since we use test set as validation, how to decide the model of best result. Just use the best result from every epoch? Or we train a fixed epoch and use the result of final epoch? Thanks!

由于临时出差取消，近两天我将完成代码的发布。基本框架不变，主要关注于bug的修复和各个trainer之间的部分代码同步。

后续暂定任务为：

• 加入CMTL的完整训练代码和部分实验结果。感谢@SelinaFelton的贡献。（4.20前完成）【已完成】
• 替换预训练网络的预训练模型加载代码，使用PyTorch官方推荐代码加载。（5.1前完成）【已完成】
• 撰写一份简单的技术报告（technical reports），简要介绍框架特性以及使用方式。（5.30前完成）【中文版正在进行】

其他任务后续在此issue中一次增加。

The C-3 framework is great for bench-marking different model and data set combinations but I'm wondering if new SotA models will be added as they're developed.
For exampled Context-Aware Crowd Counting (CAN) [Paper] [Unofficial Code] has performed amazingly well on many leader boards but I've never seen it pre-trained with GCC which may improve it even further.

There are also many other impressive models without code available such as Adaptive Scenario Discovery ASD [Paper] which would be interesting to test also.

非常感谢您的开源工作，大大减轻了我们的工作量。
现在主要有四点不懂的想向您请教，希望您不吝赐教：
1、我测试了resnet50，csrnet，VGG在shanghai_partA上的MAE，resnet50的MAE大概是110，CSRNET是72，VGG是75，似乎VGG系列在shanghai_partA上表现的更好，这与shanghai_partB不同，请问这是为什么？难道VGG系列在人群密集上的数据集特征提取能力更强？
2、CSRNET在shanghai_partA上的MAE为72，和您提供的有差距，也不及作者原论文中的68，当然我的batch size设置为1，难道是数据增强的影响吗？因为当batch>1时，会有crop并且BN的表现应该更好。
3、关于 LabelNormalize和提高学习率，您能简单介绍一下区别吗？
4、语义分割和人群计数的任务十分接近，人群计数中引入FPN这种特征金字塔将1/8的密度图输出转换到和输入相同的大小，是否会比直接上采样更好？
再次感谢您的工作，希望您能抽空解答我的问题，感激不尽。

Hi, thanks for your excellent works and discussion.

When I run the train.py for SHHA datasets, I got a dimension error(VGG net). But only happened when I run for SHHA datasets.

RuntimeError: The size of tensor a (376) must match the size of tensor b (377) at non-singleton dimension 1

Can you please give me a suggestion？

君宇学长，辛苦了，特别细心还开源了预处理的Code，之前我想上传我写的Matlab代码，今天发现跟学长的代码能力还是有很大差距的，我有两个小小的想法，想给学长汇报一下。
1）关于TBD的那些模块，我想寒假期间，跑一下实验，把我得到的结果做成跟学长统一的格式上传到github，我现在有一块1080Ti的显卡，实验速度这方面有了很大的缓解。
2）VGG和ResNet的预训练权重，为了方便大家，我想把下载链接给贴到readme.md文件里面。
最近在赶论文，任重而道远，还有很多方面要多向学长学习，哈哈。
提前给学长拜个早年，猪年行大运，诸事顺利！

您好，在参考您的代码时，如果我选择了参数量较大的模型，直接送入整张图片会有很大的计算开销，您是否考虑过将图片切片的方式呢？或者有什么更好的解决方案吗
感谢

hello, I got the density maps using test.py
now I wanna know how to get the number of people according to the density maps
thx a lot

您好。
您提供的：
https://mailnwpueducn-my.sharepoint.com/personal/gjy3035_mail_nwpu_edu_cn/_layouts/15/onedrive.aspx?id=%2Fpersonal%2Fgjy3035%5Fmail%5Fnwpu%5Fedu%5Fcn%2FDocuments%2F%E8%AE%BA%E6%96%87%E5%BC%80%E6%BA%90%E6%95%B0%E6%8D%AE%2FC3Data&originalPath=aHR0cHM6Ly9tYWlsbndwdWVkdWNuLW15LnNoYXJlcG9pbnQuY29tLzpmOi9nL3BlcnNvbmFsL2dqeTMwMzVfbWFpbF9ud3B1X2VkdV9jbi9Fa3h2T1ZKQlZ1eFBzdTc1WWZZaHY5VUJLUkZOUDdXZ0xkeFhGTVNlSEdoWGpRP3J0aW1lPWlzTnpRaFJNMTBn
可以查看，但下载的内容不正确。
另外，请问您使用的matlab的版本是多少？我用7.0执行preapre_SHHB.m，提示有语法错误。

Thanks for sharing your code.It makes life easier.I have one question that will it improve the performance if I combine multiple dataset to train the model?

Sir,I use the framework to train vgg model and SHHB,but 1000 epochs take three days. I don't know what happened？I don't modify anything

@gjy3035
hi gjy3035,

As far as I know, the spatial encoder is first applied in lane detection because of the consistency in spatial. Can you tell me some beneficial effect of this encoder applied in this crowd density estimation case? thanks.

您好！我最近尝试基于CSRNet复现WorldExpo‘10数据集上的结果，但是尚未复现到原论文的指标。原作者github没有提供相关说明。我在您的github上找到了生成WorldExpo'10数据集标签的方法是在roi区域外进行模糊处理。有几个问题咨询您一下：
（1）即使进行了模糊处理，我发现roi区域外的点也会有不清晰的人头点影响，这样做是否对训练产生不利影响？
（2）您在该数据集上训练的效果如何？对于这一数据集的训练有什么trick吗？我目前复现的效果较差的原因主要在于这一数据集的密集场景效果较差，稀疏场景正常。
我尝试用您的生成标签方法并且在lr=1e-5的条件下达到最低平均mae11.7，差于CSRNet中mae8.6效果。
非常感谢您的回复！

thanks for you code ,i want to run model of res50,SHHB got result of mae 8.3 and 13.8 with epoch 400 .but SHHA mae 109.5 mse 182.7


i found SHHB train with batch size=6 and because of different image size SHHA train with batch size=1 .res50 with some batch norm layer ,maybe question was occured here? for comparsion i run SHHB with batch size=1 got the result of mae 17.0 mse 36
only one sample with batch_norm layer may affect result .but i don t have a better idea to deal with .could you give me some advices.

@wwwzxoe303com
该网络目前仅能在pytorch0.4下训练。SANet的复现我已经基本完成。当前达到了mae12，mse19的结果。
现在还遗留一个测试代码的问题。由于作者采用的评估方案，我没有时间实现，不知你是否有时间实现？

根据作者提供的训练细节，无法训练网络。因此，我重新调整了训练参数。
config文件如下：

from easydict import EasyDict as edict
import time
import torch

# init
__C = edict()
cfg = __C

#------------------------------TRAIN------------------------
__C.SEED = 3035 # random seed,  for reporduction
__C.DATASET = 'SHHB' # dataset selection: GCC, SHHA, SHHB, UCF50, QNRF, WE

if __C.DATASET == 'UCF50':# only for UCF50
	from datasets.UCF50.setting import cfg_data
	__C.VAL_INDEX = cfg_data.VAL_INDEX 

if __C.DATASET == 'GCC':# only for GCC
	from datasets.GCC.setting import cfg_data
	__C.VAL_MODE = cfg_data.VAL_MODE 


__C.NET = 'SANet' # net selection: MCNN, VGG, VGG_DECODER, Res50, CSRNet, SANet

__C.PRE_GCC = False # use the pretrained model on GCC dataset
__C.PRE_GCC_MODEL = '' # path to model

__C.GPU_ID = [0] # sigle gpu: [0], [1] ...; multi gpus: [0,1]

# learning rate settings
__C.LR = 1e-4 # learning rate
__C.LR_DECAY = 1 # decay rate
__C.LR_DECAY_START = -1 # when training epoch is more than it, the learning rate will be begin to decay
__C.NUM_EPOCH_LR_DECAY = 1 # decay frequency
__C.MAX_EPOCH = 3000

# multi-task learning weights, no use for single model, such as MCNN, VGG, VGG_DECODER, Res50, CSRNet, and so on

__C.LAMBDA_1 = 1e-3


# print 
__C.PRINT_FREQ = 30

now = time.strftime("%m-%d_%H-%M", time.localtime())

__C.EXP_NAME = now \
			 + '_' + __C.DATASET \
             + '_' + __C.NET \
             + '_' + str(__C.LR)

if __C.DATASET == 'UCF50':
	__C.EXP_NAME += '_' + str(__C.VAL_INDEX)	

if __C.DATASET == 'GCC':
	__C.EXP_NAME += '_' + __C.VAL_MODE	

__C.EXP_PATH = './exp' # the path of logs, checkpoints, and current codes


#------------------------------VAL------------------------
__C.VAL_DENSE_START = 200
__C.VAL_FREQ = 10 # Before __C.VAL_DENSE_START epoches, the freq is set as __C.VAL_FREQ

#------------------------------VIS------------------------
__C.VISIBLE_NUM_IMGS = 1 #  must be 1 for training images with the different sizes



#================================================================================
#================================================================================
#================================================================================  

上海B的setting文件如下：

# init
__C_SHHB = edict()

cfg_data = __C_SHHB

__C_SHHB.STD_SIZE = (768,1024)
__C_SHHB.TRAIN_SIZE = (384,512)
__C_SHHB.DATA_PATH = '../ProcessedData/shanghaitech_part_B'               

__C_SHHB.MEAN_STD = ([0.452016860247, 0.447249650955, 0.431981861591],[0.23242045939, 0.224925786257, 0.221840232611])

__C_SHHB.LABEL_FACTOR = 1
__C_SHHB.LOG_PARA = 100.

__C_SHHB.RESUME_MODEL = ''#model path
__C_SHHB.TRAIN_BATCH_SIZE = 6 #imgs

__C_SHHB.VAL_BATCH_SIZE = 1 # must be 1

Hi @gjy3035
I have trained the model VGG and resnet101-SFCN on my dataset. They didn't converge with the increasing training epochs. The best result for VGG is achieved at epoch 1, and the best result for resnet101-SFCN is obtained at epoch 2. Did you have this experience in your training results? Thank you.

How to calculate the number of people in one picture?

Can I just sum the normalized pred_map in your test.py?

for exmaple:
`pred_map = net.test_forward(img)
pred_map = pred_map.cup().data.numpy()[0,0,:,:]

pred = np.sum(pred_map)/100.0
pred_map = pred_map/np.max(pred_map+1e-20)

count = np.sum(pred_map)
`

你好，为什么VGG16选择使用conv4_3的输出，ResNet选择使用layer3的输出呢？为什么不选择conv5_3和layer4的输出呢？也就是说，在输出feature map的深度和空间尺寸上怎么权衡呢？

Dear Author, thank you fro providing this awesome framework. I have trained the Res50 model with ShangHaiTechB dataset but only get MAE with 8.4 instead of 7.7 (reported), is there any hyperparameter setting that required?