zengxh / dmm_net Goto Github PK

The extracted proposals can't be downloaded.
Besides, youtubeVOS-2018 dataset doesn't contains 'train_testdev_ot'

[Errno 2] No such file or directory: './experiments/proposals/coco81/inference/youtubevos_train3k_meta/video/countobj.json'
我在终端运行sh train_101.sh时，出现没有这个文件，下载的coco81里面没有，这个怎么解决

Hi, I'm trying to run the model but in the training part, I can't find the file scripts/train/train_load_prop.sh to train the model. Or is it correct to use the file train.py?

Hi,this link of extracted proposals is broken http://www.cs.toronto.edu/~xiaohui/dmm/proposals/proposals_ytb_ot.tar.gz

I have another ticket open here
z-x-yang/CFBI#10

They've created something similar to this repo.

It seems doable - need spit out individual pngs from video

ffmpeg -i VIDEOHERE.mov %04d_img.png -hide_banner

but the way the code is tangled with meta.json makes things a tad difficult. Is there a shortcut or any scripts you can provide?

I have followed every step in the DMM net installation exactly. My cuda is 10.0 and Cudnn 7.6.5.

I get this error, when I am running the eval_r50.sh?

AttributeError: module 'torch._six' has no attribute 'PY3'

Hi！Can you provide the proposals and codes for DAVIS dataset? Thanks!

Hi! I follow your instruction but get wrong J and F scores, can you give me some advice?

[INFO][26-11-2019 21:31:09] Evaluating measure: J
[INFO][26-11-2019 21:31:17] J mean 0.06769988522362873
[INFO][26-11-2019 21:31:17] J recall 0.020832281616595345
[INFO][26-11-2019 21:31:17] J decay -0.022702139123339328
[INFO][26-11-2019 21:31:17] Evaluating measure: F
[INFO][26-11-2019 21:32:45] F mean 0.10614527875319188
[INFO][26-11-2019 21:32:45] F recall 0.028320632242200872
[INFO][26-11-2019 21:32:45] F decay -0.01829959926218509

Hi, I have disabled the distributed training when I use multi-GPU training. But there appears a bug, which point to the different value between "len(proposal_cur)" and "x.shape[0]" in trainer.py. I have printed the value of "len(proposal_cur)" and "x.shape[0]" as below. By the way, this bug didn't appear when I use single GPU training.
Can you help me to figure this bug out? Thanks a lot!

(dmm) $:~/Experiments/DMM_Net$ sh scripts/train/train_101.sh
2020-05-12 08:23:18,754-{train.py:394}-INFO-[model_name] ytb_train_x101
2020-05-12 08:23:18,755-{train.py:395}-INFO-get number of gpu: 3
2020-05-12 08:23:20,180-{utils.py:213}-INFO-[load_DMM_config] dmm/configs/train.yaml
2020-05-12 08:23:20,185-{utils.py:232}-INFO-ud relax_max_iter 400 -> 10|ud relax_proj_iter 50 -> 5
2020-05-12 08:23:25,200-{utils.py:213}-INFO-[load_DMM_config] dmm/configs/train.yaml
2020-05-12 08:23:25,208-{utils.py:232}-INFO-ud relax_max_iter 400 -> 10|ud relax_proj_iter 50 -> 5
2020-05-12 08:23:25,213-{train.py:162}-INFO-{'sort_max_num': 50, 'matching_score_thre': 0.0, 'score_weight': 0.3, 'relax': 1, 'relax_max_iter': 10, 'relax_proj_iter': 5, 'relax_topk': 0, 'relax_learning_rate': 0.1, 'matching': {'match_max_score': 1, 'algo': 'relax', 'cost': 'cosine'}, 'encoder': {'nms_thresh': 0.4}}
2020-05-12 08:23:25,342-{trainer.py:63}-INFO-load from json_data; num vid 3000
2020-05-12 08:23:25,342-{train.py:164}-INFO-init model 6.586
2020-05-12 08:23:25,344-{train.py:171}-INFO-optimizer 0.002
2020-05-12 08:23:25,344-{train.py:173}-INFO-[enc_opt] len: 2; len for each param group: [48, 314]
2020-05-12 08:23:25,344-{train.py:175}-INFO-[dec_opt] len: 1; len for each param group: [10]
2020-05-12 08:23:25,352-{train.py:222}-INFO-save args in experiments/models/ytb_train_x101/05-12-08-23args.pkl
2020-05-12 08:23:25,352-{train.py:223}-INFO-Namespace(augment=True, base_model='resnet101', batch_size=4, best_val_loss=0, cache_data=1, config_train='dmm/configs/train.yaml', dataset='youtube', davis_eval_folder='', device=device(type='cuda', index=0), distributed=0, distributed_manully=0, distributed_manully_Nrep=0, distributed_manully_rank=0, dropout=0.0, epoch_resume=0, eval_flag='pred', eval_split='trainval', finetune_after=3, gpu_id=0, gt_maxseqlen=5, hidden_size=128, imsize=480, iou_weight=1.0, kernel_size=3, length_clip=3, load_proposals=1, load_proposals_dataset=1, local_rank=0, log_file='train.log', log_term=False, loss_weight_iouraw=1.0, loss_weight_match=1.0, lr=0.0001, lr_cnn=1e-05, lr_decoder=0.001, mask_th=0.5, max_dets=100, max_epoch=2, max_eval_iter=800, maxseqlen=5, min_delta=0.0, min_size=0.001, model_dir='experiments/models/ytb_train_x101', model_name='ytb_train_x101', models_root='experiments/models/', momentum=0.9, my_augment=False, ngpus=3, num_classes=21, num_workers=4, only_spatial=False, only_temporal=False, optim='adam', optim_cnn='adam', overwrite_loadargs=1, pad_video=0, patience=15, patience_stop=60, pred_offline_meta='../data/ytb_vos/splits_813_3k_trainvaltest/meta_vid_frame_2_predid.json', pred_offline_path=['./experiments/proposals/coco81/inference/youtubevos_train3k_meta/asdict_50/videos/'], pred_offline_path_eval=['experiments/proposals/coco81/inference/youtubevos_val200_meta/asdict_50/pred_DICT.pth'], prev_mask_d=1, print_every=2, random_select_frames=1, resize=True, resume=False, resume_path='epoxx_iterxxxx', rotation=10, sample_inference_mask=0, save_every=3000, seed=123, shear=0.1, single_object=False, skip_empty_starting_frame=1, skip_mode='concat', test=0, test_image_h=256, test_image_w=448, test_model_path='', threshold_mask=0.4, train_h=255, train_split='train', train_w=448, translation=0.1, update_encoder=1, use_gpu=True, use_refmask=0, weight_decay=1e-06, weight_decay_cnn=1e-06, year='2017', youtube_dir='../../databases/YouTubeVOS/', zoom=0.7)
2020-05-12 08:23:25,353-{train.py:232}-INFO-init_dataloaders
2020-05-12 08:23:25,527-{youtubeVOS.py:84}-INFO-[dataset] phase read train; len of db seq 3000
2020-05-12 08:23:25,527-{youtubeVOS.py:103}-INFO-LMDB not found. This could affect the data loading time. It is recommended to use LMDB.
2020-05-12 08:23:25,527-{youtubeVOS.py:115}-INFO-no cache data found at data/ytb_vos/splits_813_3k_trainvaltest/dmm_cached_train.pkl; it will take a while to cache the data
2020-05-12 08:29:53,807-{youtubeVOS.py:121}-INFO-try to dump in data/ytb_vos/splits_813_3k_trainvaltest/dmm_cached_train.pkl
2020-05-12 08:29:56,467-{dataset.py:125}-INFO-+new_parts 200: 1.6958227157592773
2020-05-12 08:30:07,352-{dataset.py:125}-INFO-+new_parts 200: 12.146138191223145
2020-05-12 08:30:58,266-{youtubeVOS.py:125}-INFO-load lmdb 452.77
2020-05-12 08:31:22,048-{youtubeVOS.py:161}-INFO-filtered images out -> 444 for #vid 3000
2020-05-12 08:31:23,869-{youtubeVOS.py:253}-INFO-[init][data][youtube][load clips] load anno 25.58; cliplen 3| annotation clip 26261(skip 59)| videos 3000
2020-05-12 08:31:24,195-{youtubeVOS.py:265}-INFO-load keys 0.33
2020-05-12 08:31:24,196-{train.py:104}-INFO-INPUT shape: 255 448
2020-05-12 08:31:24,316-{dataset.py:119}-INFO-[trainval] loading offline from experiments/proposals/coco81/inference/youtubevos_val200_meta/asdict_50/pred_DICT.pth; Nf ['experiments/proposals/coco81/inference/youtubevos_val200_meta/asdict_50/pred_DICT.pth']
2020-05-12 08:31:36,170-{dataset.py:125}-INFO-+new_parts 200: 11.853206157684326
2020-05-12 08:31:36,178-{dataset.py:133}-INFO-load offline use 11.86 | len 200
2020-05-12 08:31:36,180-{youtubeVOS.py:84}-INFO-[dataset] phase read trainval; len of db seq 200
2020-05-12 08:31:36,196-{youtubeVOS.py:103}-INFO-LMDB not found. This could affect the data loading time. It is recommended to use LMDB.
2020-05-12 08:31:36,196-{youtubeVOS.py:115}-INFO-no cache data found at data/ytb_vos/splits_813_3k_trainvaltest/dmm_cached_trainval.pkl; it will take a while to cache the data
2020-05-12 08:31:57,003-{youtubeVOS.py:121}-INFO-try to dump in data/ytb_vos/splits_813_3k_trainvaltest/dmm_cached_trainval.pkl
2020-05-12 08:31:57,383-{youtubeVOS.py:125}-INFO-load lmdb 21.20
2020-05-12 08:31:57,399-{youtubeVOS.py:161}-INFO-filtered images out -> 0 for #vid 200
2020-05-12 08:31:57,421-{youtubeVOS.py:253}-INFO-[init][data][youtube][load clips] load anno 0.04; cliplen 3| annotation clip 800(skip 6)| videos 200
2020-05-12 08:31:57,424-{youtubeVOS.py:265}-INFO-load keys 0.00
2020-05-12 08:31:57,425-{train.py:104}-INFO-INPUT shape: 255 448
2020-05-12 08:31:57,425-{train.py:237}-INFO-dataloader 512.072
2020-05-12 08:31:57,427-{train.py:258}-INFO-epoch 0 - trainval;
2020-05-12 08:31:57,427-{train.py:260}-INFO--- loss weight loss_weight_match: 1.0 loss_weight_iouraw 1.0;
Traceback (most recent call last):
File "train.py", line 413, in
trainIters(args)
File "train.py", line 285, in trainIters
loss, losses = trainer(batch_idx, inputs, imgs_names, targets, seq_name, starting_frame, split, args, proposals)
File "/home/csy/anaconda3/envs/dmm/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/home/csy/anaconda3/envs/dmm/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 152, in forward
outputs = self.parallel_apply(replicas, inputs, kwargs)
File "/home/csy/anaconda3/envs/dmm/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 162, in parallel_apply
return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
File "/home/csy/anaconda3/envs/dmm/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py", line 83, in parallel_apply
raise output
File "/home/csy/anaconda3/envs/dmm/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py", line 59, in _worker
output = module(*input, **kwargs)
File "/home/csy/anaconda3/envs/dmm/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/home/csy/Experiments/DMM_Net/dmm/modules/trainer.py", line 112, in forward
CHECKEQ(len(proposal_cur), x.shape[0])
File "/home/csy/Experiments/DMM_Net/dmm/utils/checker.py", line 27, in CHECKEQ
assert(a == b), 'get {} {}'.format(a, b)
AssertionError: get 2 1
len(proposal_cur): 2
x.shape[0]: 1
len(proposal_cur): 2
x.shape[0]: 1

when i run python train.py, the following error occurs:

2019-10-16 09:48:22,325-{train.py:384}-INFO-[model_name] ytb_r50_w11
2019-10-16 09:48:22,325-{train.py:385}-INFO-get number of gpu: 1
2019-10-16 09:48:23,989-{utils.py:213}-INFO-[load_DMM_config] dmm/configs/train.yaml
2019-10-16 09:48:23,996-{utils.py:232}-INFO-ud relax_max_iter 400 -> 10|ud relax_proj_iter 50 -> 5
2019-10-16 09:48:24,041-{utils.py:213}-INFO-[load_DMM_config] dmm/configs/train.yaml
2019-10-16 09:48:24,047-{utils.py:232}-INFO-ud relax_max_iter 400 -> 10|ud relax_proj_iter 50 -> 5
2019-10-16 09:48:24,050-{train.py:152}-INFO-{'sort_max_num': 50, 'matching_score_thre': 0.0, 'score_weight': 0.3, 'relax': 1, 'relax_max_iter': 10, 'relax_proj_iter': 5, 'relax_topk': 0, 'relax_learning_rate': 0.1, 'matching': {'match_max_score': 1, 'algo': 'relax', 'cost': 'cosine'}, 'encoder': {'nms_thresh': 0.4}}
2019-10-16 09:48:26,367-{trainer.py:63}-INFO-load from json_data; num vid 3000
2019-10-16 09:48:26,367-{train.py:154}-INFO-init model 4.042
2019-10-16 09:48:26,369-{train.py:161}-INFO-optimizer 0.001
2019-10-16 09:48:26,369-{train.py:163}-INFO-[enc_opt] len: 2; len for each param group: [48, 161]
2019-10-16 09:48:26,369-{train.py:165}-INFO-[dec_opt] len: 1; len for each param group: [10]
2019-10-16 09:48:26,371-{train.py:213}-INFO-save args in experiments/ytb_r50_w11/10-16-09-48args.pkl
2019-10-16 09:48:26,371-{train.py:214}-INFO-Namespace(augment=False, base_model='resnet50', batch_size=4, best_val_loss=0, cache_data=1, config_train='dmm/configs/train.yaml', dataset='youtube', davis_eval_folder='', device=device(type='cuda', index=0), distributed=0, distributed_manully=0, distributed_manully_Nrep=0, distributed_manully_rank=0, dropout=0.0, epoch_resume=0, eval_flag='pred', eval_split='trainval', finetune_after=0, gpu_id=0, gt_maxseqlen=5, hidden_size=128, imsize=480, iou_weight=1.0, kernel_size=3, length_clip=3, load_proposals=1, load_proposals_dataset=1, local_rank=0, log_file='train.log', log_term=False, loss_weight_iouraw=18.0, loss_weight_match=1.0, lr=0.001, lr_cnn=0.0001, lr_decoder=0.001, mask_th=0.5, max_dets=100, max_epoch=100, max_eval_iter=800, maxseqlen=5, min_delta=0.0, min_size=0.001, model_dir='experiments/ytb_r50_w11', model_name='ytb_r50_w11', models_root='experiments/', momentum=0.9, my_augment=False, ngpus=1, num_classes=21, num_workers=4, only_spatial=False, only_temporal=False, optim='adam', optim_cnn='adam', overwrite_loadargs=1, pad_video=0, patience=15, patience_stop=60, pred_offline_meta='data/ytb_vos/splits_813_3k_trainvaltest/meta_vid_frame_2_predid.json', pred_offline_path=['experiments/proposals/coco81/inference/youtubevos_val200_meta/asdict_50/pred_DICT.pth'], pred_offline_path_eval=None, prev_mask_d=1, print_every=2, random_select_frames=0, resize=False, resume=False, resume_path='epoxx_iterxxxx', rotation=10, sample_inference_mask=0, save_every=3000, seed=123, shear=0.1, single_object=False, skip_empty_starting_frame=0, skip_mode='concat', test=0, test_image_h=256, test_image_w=448, test_model_path='', threshold_mask=0.4, train_h=255, train_split='train', train_w=448, translation=0.1, update_encoder=1, use_gpu=True, use_refmask=0, weight_decay=1e-06, weight_decay_cnn=1e-06, year='2017', youtube_dir='../../databases/YouTubeVOS/', zoom=0.7)
2019-10-16 09:48:26,372-{train.py:223}-INFO-init_dataloaders
2019-10-16 09:48:26,412-{dataset.py:119}-INFO-[train] loading offline from experiments/proposals/coco81/inference/youtubevos_val200_meta/asdict_50/pred_DICT.pth; Nf ['experiments/proposals/coco81/inference/youtubevos_val200_meta/asdict_50/pred_DICT.pth']
2019-10-16 09:48:27,298-{dataset.py:125}-INFO-+new_parts 200: 0.8864507675170898
2019-10-16 09:48:27,303-{dataset.py:133}-INFO-load offline use 0.89 | len 200
2019-10-16 09:48:27,320-{youtubeVOS.py:84}-INFO-[dataset] phase read train; len of db seq 3000
2019-10-16 09:48:27,320-{youtubeVOS.py:103}-INFO-LMDB not found. This could affect the data loading time. It is recommended to use LMDB.
2019-10-16 09:48:27,321-{youtubeVOS.py:115}-INFO-no cache data found at data/ytb_vos/splits_813_3k_trainvaltest/dmm_cached_train.pkl; it will take a while to cache the data
2019-10-16 10:17:41,177-{youtubeVOS.py:121}-INFO-try to dump in data/ytb_vos/splits_813_3k_trainvaltest/dmm_cached_train.pkl
2019-10-16 10:18:03,726-{youtubeVOS.py:125}-INFO-load lmdb 1776.42
Traceback (most recent call last):
File "/home/zhanglin/Research/codes/2020/DMM_Net/train.py", line 403, in
trainIters(args)
File "/home/zhanglin/Research/codes/2020/DMM_Net/train.py", line 225, in trainIters
loaders = init_dataloaders(args)
File "/home/zhanglin/Research/codes/2020/DMM_Net/train.py", line 86, in init_dataloaders
use_prev_mask = False)
File "/home/zhanglin/Research/codes/2020/DMM_Net/dmm/dataloader/dataset_utils.py", line 17, in get_dataset
use_prev_mask = use_prev_mask)
File "/home/zhanglin/Research/codes/2020/DMM_Net/dmm/dataloader/youtubeVOS.py", line 157, in init
images_valid = [fname for img, fname in zip(images, seq.files) if self.countobj[dbname][img] > 0 ]
File "/home/zhanglin/Research/codes/2020/DMM_Net/dmm/dataloader/youtubeVOS.py", line 157, in
images_valid = [fname for img, fname in zip(images, seq.files) if self.countobj[dbname][img] > 0 ]
KeyError: '003234408d'

Process finished with exit code 1

i don't know why....looking forward to your reply