captaineven / mcmot-bytetrack Goto Github PK

@CaptainEven hi thanks for sharing the code base i had following queries

1. had reid been added as an additional head to the yolox architecture ?
2. If so can you share the location of the addition module
3. the current repo does tracking for 5 classes ? does it do reid for those 5 classes also ?

Thnaks for the response

Hi,
Can you please guide for the following queries:

1. I have the output of detector in .txt file, in which part of the code I should mention the class ID indexing so that it should pick the specific number of classes?
2. Do I need to create the .json file for multi-class? If yes, then in which part of the code do I have to mention the class ids? Let's say I am following the MOT-17 format and trying to use only test data.
3. Can I run the tracker without yolox pre-trained file? Means, simply giving the detection as input to the ByteTrack for tracking for multiple classes.

I shall be thankful for your response.

Kind Regards,

Hello
How are you?
Thanks for contributing to this project.
This project shows a simple explanation in combining this tracker with other detector.
But I think that the detailed detector specification is needed to combine with this tracker.
That's because a general detector filters objects with confidences lower than a certain threshold in post-processing like NMS.
I found that this tracker uses a high threshold and a low threshold for detection confidence.
I think that the logic using confidences in a detector and this tracker should be consistent.

Can you show me the format of your training list and some details of your training?

It seems the chechpoint file worked very well in your demo video (which belong to C5 dataset), but not working great in VisDrone dataset. Can you pls tell us where to download the C5 Dataset, cause I really cannot find this dataset .

Thanks for your kindness

Hi CaptainEven, I appreciated your work. However, I had trouble downloading weight (checkpoint). I find it very hard to comprehend the website (baidu) since English is the only language I know. Can you upload your weight to Onedrive or Google drive, also, can I use my own weight? If I can, what I should do? Thank you very much

I am trying to train with the following commands.

! wget https://github.com/Megvii-BaseDetection/storage/releases/download/0.0.1/yolox_tiny_32dot8.pth -P ../YOLOX_outputs/yolox_tiny_track_c5_darknet/
! python3 train_det.py -d 0 -b 16 -o --data_dir ../yolox/data/datasets/coco128 -c ../YOLOX_outputs/yolox_tiny_track_c5_darknet/yolox_tiny_32dot8.pth

However, an error occurs.

[Err]: invalid file list path!

Where can I find the following files?

train_f_list_path '/users/duanyou/c5/data_all/train_all.txt'
test_f_list_path '/users/duanyou/c5/data_all/test5000.txt'

Please advise me.

Hi,

When I initialize this project and try to run tool/demo_mcmot.py, I got this trackbace no module name yolox_tiny_track_c5_darknet.

What can I do to solve this.

Thanks

I excepted a multi-class multi-object tracking.
However, the output format is save_format = '{frame},{id},{x1},{y1},{w},{h},{s},-1,-1,-1\n'
which takes no account of the category id of the object, like in your YOLOV4_MCMOT.

Is this MCMOT or MOT?

@CaptainEven Thanks for sharing your code, can you teach me how to convert Visdrone-MOT dataset and train on this repo? Thank you.

hi @CaptainEven

could you give a bit more guidance on how to run the demo with REID on the 5 classes pretrained model ?
thanks

Can you help me to train custom datasets with more than 5 classes? I've tried train_track.py but got this error:

2022-05-17 14:38:21 | INFO     | yolox.data.datasets.mcmotdataset:37 - Classes: car, bicycle, person, cyclist, tricycle
2022-05-17 14:38:21 | ERROR    | yolox.data.datasets.mcmotdataset:48 - Invalid img root root: ..\dataset\train/JPEGImages

But, my dataset's classes are different what it mentioned. How to fix this? I think it's because of dataset load failure.

Still got error "Unexpected key(s) in state_dict: "mtl_loss.sigma", after change branch to DarknetCfg, and use the latest checkpoint

Here is my command lin input and output info:

(mcbytetrack) xys@a325xys:/Downloads/zhaoqyu/code/MCMOT-ByteTrack_upd$ git checkout DarknetCfg
D datasets/data_path/citypersons.train
D datasets/data_path/eth.train
D datasets/max_id_dict.npz
M exps/example/mot/yolox_tiny_track_c5_darknet.py
M tools/demo_mcmot.py
Already on 'DarknetCfg'
Your branch is up to date with 'origin/DarknetCfg'.
(mcbytetrack) xys@a325xys:/Downloads/zhaoqyu/code/MCMOT-ByteTrack_upd$ python3 tools/demo_mcmot.py --reid True -f exps/example/mot/yolox_tiny_track_c5_darknet.py -c /home/xys/Downloads/zhaoqyu/code/MCMOT-ByteTrack_upd/pretrained/latest_ckpt.pth.tar --path /home/xys/Downloads/zhaoqyu/code/MCMOT-ByteTrack/datasets/tt.mp4
2022-03-23 20:35:23.957 | INFO | yolox.exp.build:get_exp_by_file:17 - Exp file path: /home/xys/Downloads/zhaoqyu/code/MCMOT-ByteTrack_upd/exps/example/mot/yolox_tiny_track_c5_darknet.py.
2022-03-23 20:35:23.957 | INFO | yolox.exp.build:get_exp_by_file:20 - Exp file's dir path: /home/xys/Downloads/zhaoqyu/code/MCMOT-ByteTrack_upd/exps/example/mot.
2022-03-23 20:35:23.957 | INFO | yolox.exp.build:get_exp_by_file:25 - Module name: yolox_tiny_track_c5_darknet.
{0: 1266, 1: 313, 2: 490, 3: 758, 4: 82}
ReID: False
Number of classes: 5
2022-03-23 20:35:23.958 | INFO | main:run:586 - Args: Namespace(camid=0, ckpt='/home/xys/Downloads/zhaoqyu/code/MCMOT-ByteTrack_upd/pretrained/latest_ckpt.pth.tar', class_names=['car', ' bicycle', ' person', ' cyclist', ' tricycle'], conf=None, debug=True, demo='video', device='gpu', exp_file='exps/example/mot/yolox_tiny_track_c5_darknet.py', experiment_name='yolox_tiny_track_c5_darknet', fp16=False, fuse=False, match_thresh=0.8, min_box_area=10, mot20=False, n_classes=5, name=None, nms=None, path='/home/xys/Downloads/zhaoqyu/code/MCMOT-ByteTrack/datasets/tt.mp4', reid=True, save_result=True, task='track', track_buffer=240, track_thresh=0.5, trt=False, tsize=None, video_dir='../videos')
2022-03-23 20:35:23.961 | INFO | yolox.models.yolox_darknet:init:753 - Network config file parsed.
2022-03-23 20:35:24.007 | INFO | yolox.models.yolox_darknet:init:756 - Network modules built.
2022-03-23 20:35:24.007 | INFO | yolox.models.yolox_darknet:init:760 - Network weights initialized.
2022-03-23 20:35:24.007 | INFO | yolox.models.yolox_darknet:init:766 - Number of object classes: 5.
Net size: (768, 448)
2022-03-23 20:35:24.007 | INFO | yolox.models.yolox_darknet:init:773 - ReID: True
2022-03-23 20:35:25.498 | INFO | main:run:614 - Loading checkpoint...
Traceback (most recent call last):
File "tools/demo_mcmot.py", line 662, in
run(exp, args)
File "tools/demo_mcmot.py", line 618, in run
model.load_state_dict(ckpt["model"])
File "/home/xys/anaconda3/envs/mcbytetrack/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1497, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for YOLOXDarknetReID:
Unexpected key(s) in state_dict: "mtl_loss.sigma".
(mcbytetrack) xys@a325xys:~/Downloads/zhaoqyu/code/MCMOT-ByteTrack_upd$

Hi,
where can I get max_id_dict.npz?

Hi Thank you for this code, but I can see some code errors in https://github.com/CaptainEven/MCMOT-ByteTrack/blob/DarknetCfg/tutorials/fairmot/tracker.py (line 255-274)

In line 258:

dets, inds = mot_decode(hm, wh, reg=reg, ltrb=self.opt.ltrb, K=self.opt.K)

If self.opt.K = 500 for instance, the sizes of dets and inds are 500.

Then in line 263 - 264:

dets = self.post_process(dets, meta)
dets = self.merge_outputs([dets])[1]

The dets are separated and process per class. lets say 4 class, hence the len(dets) == num_classes, but the inds is still 500

Then, in line 266-272:

remain_inds = dets[:, 4] > self.opt.conf_thres
inds_low = dets[:, 4] > 0.2
#inds_low = dets[:, 4] > self.opt.conf_thres
inds_high = dets[:, 4] < self.opt.conf_thres
inds_second = np.logical_and(inds_low, inds_high)
dets_second = dets[inds_second]
id_feature_second = id_feature[inds_second]

These indexes do not match for slicing.
IndexError: boolean index did not match indexed array along dimension 0; dimension is 500 but corresponding boolean dimension is 179

The inds_second has length of just one class, not entire detection's id_feature. The id_feature did not go through the class ordering in the post_process as the dets.

I think on line 264 dets = self.merge_outputs([dets])[1] you are selecting only the first class. So objects belonging other classes are not tracked.

@CaptainEven can you pls share the weight file on google drive link ?

可以用来跟踪其他物体吗？比如无人船这种？

i only see ReID is used in bytetrack? Does ocsort also can use ReID?

@CaptainEven thanks for sharing the code base had few queries

1. the current code base supports reid for only person ?
2. can we extend this reid principle to objects and vehicles also ? if so what all changes have to be made

Thanks in advance

Hello
How are you?
Thanks for contributing to this project.
I found that this method does NOT estimate new position of object without detection of object.
In general, there are cases when there is NOT a detection result of an object at certain moment.
I think that new position estimation of an object should be done in even such case.

when i want run the demo file python demo_mcmot.py, i get this message:

2022-03-20 17:34:04.333 | INFO     | yolox.exp.build:get_exp_by_file:17 - Exp file path: /home/nicken/ai/cv_task/Object_Tracking/repository/MCMOT-ByteTrack/exps/example/mot/yolox_tiny_track_c5_darknet.py.
2022-03-20 17:34:04.333 | INFO     | yolox.exp.build:get_exp_by_file:20 - Exp file's dir path: /home/nicken/ai/cv_task/Object_Tracking/repository/MCMOT-ByteTrack/exps/example/mot.
2022-03-20 17:34:04.333 | INFO     | yolox.exp.build:get_exp_by_file:25 - Module name: yolox_tiny_track_c5_darknet.
{0: 1266, 1: 313, 2: 490, 3: 758, 4: 82}
ReID:  False
Number of classes:  5
2022-03-20 17:34:04.338 | INFO     | __main__:run:586 - Args: Namespace(camid=0, ckpt='../latest_ckpt.pth.tar', class_names=['car', ' bicycle', ' person', ' cyclist', ' tricycle'], conf=None, debug=True, demo='video', device='gpu', exp_file='../exps/example/mot/yolox_tiny_track_c5_darknet.py', experiment_name='yolox_tiny_track_c5_darknet', fp16=False, fuse=False, match_thresh=0.8, min_box_area=10, mot20=False, n_classes=5, name=None, nms=None, path='../videos/test_13.mp4', reid=False, save_result=True, task='track', track_buffer=240, track_thresh=0.5, trt=False, tsize=None, video_dir='../videos')
2022-03-20 17:34:04.342 | INFO     | yolox.models.yolox_darknet:__init__:753 - Network config file parsed.
2022-03-20 17:34:04.407 | INFO     | yolox.models.yolox_darknet:__init__:756 - Network modules built.
2022-03-20 17:34:04.407 | INFO     | yolox.models.yolox_darknet:__init__:760 - Network weights initialized.
2022-03-20 17:34:04.407 | INFO     | yolox.models.yolox_darknet:__init__:766 - Number of object classes: 5.
Net size:  (768, 448)
2022-03-20 17:34:04.407 | INFO     | yolox.models.yolox_darknet:__init__:773 - ReID: True
/home/nicken/.pyenv/versions/pytorch/lib/python3.7/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at  /pytorch/c10/core/TensorImpl.h:1156.)
  return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
2022-03-20 17:34:06.095 | INFO     | __main__:run:601 - Model Summary: Params: 0.00M, Gflops: 0.00
2022-03-20 17:34:06.112 | INFO     | __main__:run:614 - Loading checkpoint...
Traceback (most recent call last):
  File "demo_mcmot.py", line 662, in <module>
    run(exp, args)
  File "demo_mcmot.py", line 618, in run
    model.load_state_dict(ckpt["model"])
  File "/home/nicken/.pyenv/versions/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1407, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for YOLOXDarknetReID:
        Unexpected key(s) in state_dict: "mtl_loss.sigma". 

run cmd: python demo_mcmot.py -c ../latest_ckpt.pth.tar

can you tell want happen？I want run this code to success in video

Let me mention that training on dataset with 6K inputs and 1 class, works great. However the training with 300 classes and 6000 images large dateset causes the following error:

------------CPU Mode for This Batch-------------
2022-05-25 13:52:53 | INFO     | yolox.models.yolo_head:335 - OOM RuntimeError is raised due to the huge memory cost during label assignment. 
CPU mode is applied in this batch. If you want to avoid this issue, try to reduce the batch size or image size.
OOM RuntimeError is raised due to the huge memory cost during label assignment. 
CPU mode is applied in this batch. If you want to avoid this issue, try to reduce the batch size or image size.

Training continues for some time, then quits completely with "CUDA out of memory":

RuntimeError: CUDA out of memory. Tried to allocate 22.00 MiB (GPU 0; 7.79 GiB total capacity; 6.20 GiB already allocated; 21.44 MiB free; 6.26 GiB reserved in total by PyTorch)

Trying to fix it. Help from everyone else is welcome. Please drop your solutions/thoughts here. As soon as I find a solution, I will share it here too. Thank you.

I'm running on 2 GPUs. GTX 1070 and RTX 3070. Should be plenty. Platform needs more optimization.

P.S. Lowering batch doesn't help. I set batch to "-b 2" and devices also to "-d 2". 1 batch per GPU. Can't get lower than that.

Image size is set to:
self.input_size = (256, 512)
self.test_size = (256, 512)
Also very low res.

I try to run demo track with this code
python3 tools/demo_track.py -f exps/example/mot/yolox_x_mix_det.py -c pretrained/bytetrack_x_mot17.pth.tar --fp16 --fuse --save_result /content/MCMOT-ByteTrack
but get this error

Traceback (most recent call last):
  File "tools/demo_track.py", line 480, in <module>
    main(exp, args)
  File "tools/demo_track.py", line 442, in main
    model.load_state_dict(ckpt["model"])
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1483, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for YOLOX:
	size mismatch for head.cls_preds.0.weight: copying a param with shape torch.Size([1, 320, 1, 1]) from checkpoint, the shape in current model is torch.Size([5, 320, 1, 1]).
	size mismatch for head.cls_preds.0.bias: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([5]).
	size mismatch for head.cls_preds.1.weight: copying a param with shape torch.Size([1, 320, 1, 1]) from checkpoint, the shape in current model is torch.Size([5, 320, 1, 1]).
	size mismatch for head.cls_preds.1.bias: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([5]).
	size mismatch for head.cls_preds.2.weight: copying a param with shape torch.Size([1, 320, 1, 1]) from checkpoint, the shape in current model is torch.Size([5, 320, 1, 1]).
	size mismatch for head.cls_preds.2.bias: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([5]).

Is it possible to get the checkpoints into a Google drive? I can’t access the link in the readme. It would be highly appreciated!

Thank you for your work!