How to apply code into kitti tracking test set? about 3d-vehicle-tracking HOT 5 CLOSED

I have the same question with applying kitti dataset. I really doubt the validity of this code. Have you solved it? Looking forward to reply!

from 3d-vehicle-tracking.

I have the same question with applying kitti dataset. I really doubt the validity of this code. Have you solved it? Looking forward to reply!

Hi,

I didn't get the response from the author, but I find the generated file such as kf3d_age20_aff0.1_hit0_100m_803_pd.json actually includes information like position, dimension and depth, etc.

But the result seems not very good on KITTI test set, I'm not sure if I understand the information correctly.

here is the visualized result:

another method:

this method:

Hope it helps.

from 3d-vehicle-tracking.

exo me? where did you download the kitti tracking dataset? from kitti_tracking @chrisHuxi or other places?

from 3d-vehicle-tracking.

exo me? where did you download the kitti tracking dataset? from kitti_tracking @chrisHuxi or other places?

yes exactly

from 3d-vehicle-tracking.

Hi @chrisHuxi, you are on the right track. Just a kind reminder that some of the scripts are not for testing set evaluation.

Hi, thanks for sharing your great job.

I'm trying to apply code into kitti tracking test set, but it seems doesn't work, did I miss something?

Here is the steps I did:

1. I put the calib, image, oxts file into folder: 3d-vehicle-tracking/3d-tracking/data/kitti_tracking/testing, and I create empty folder 0000, 0001, ... ,0028 in label_02. so the structure looks like:

calib/
calib/0000.txt
...
image_02/
image_02/0000/000000.png
...
label_02/
label_02/0000/            <== empty
...
oxts/0000.txt
...

1. I downloaded the kitti_test_trk_detections_RCC.pkl, put it into folder: 3d-vehicle-tracking/3d-tracking/data/kitti_tracking/ and rename it as kitti_test_trk_detections.pkl
2. run script:
   PYTHONPATH=. python loader/gen_dataset.py kitti test --kitti_task track --mode test
   Then I got empty labels in folder label_02
3. run script:
   PYTHONPATH=. python loader/gen_pred.py kitti test
   Then I got result:

 1550 images.
 Frame 1550, GT: 0 Boxes, PD: 3 Boxes
 Frame 1551, GT: 0 Boxes, PD: 2 Boxes
 Frame 1552, GT: 0 Boxes, PD: 2 Boxes
 Frame 1553, GT: 0 Boxes, PD: 3 Boxes
 Frame 1554, GT: 0 Boxes, PD: 2 Boxes
 Frame 1555, GT: 0 Boxes, PD: 3 Boxes
 Frame 1556, GT: 0 Boxes, PD: 3 Boxes
 Frame 1557, GT: 0 Boxes, PD: 2 Boxes
 Frame 1558, GT: 0 Boxes, PD: 2 Boxes
 Frame 1559, GT: 0 Boxes, PD: 2 Boxes

Here I assume that GT is the label, because we use the empty as label, so it always return 0 BBoxes. And the PD means prediction result. So far I think everything works fine.

1. run script:
   PYTHONPATH=. python run_estimation.py kitti test --session 623 --epoch 100
   Then I got msg:

GT is empty
GT is empty
GT is empty
GT is empty
GT is empty
GT is empty
GT is empty
GT is empty
GT is empty
Prediction is empty
GT is empty
Prediction is empty
GT is empty
Prediction is empty

Here I don't understand why the prediction is empty sometimes <== not every frame, mostly it only shows "GT is empty"
and it generate a new folder : 3d-tracking/output/623_100_kitti_test_set, and some ***output.pkl file

GT can be empty, since the annotation of the testing set is not pubicly available. PD, which is prediction, can also be empty, due to absence of objects, filtered or poor object detection results.

6. PYTHONPATH=. python run_tracking.py kitti test --session 623 --epoch 100

Writing to output/623_100_kitti_test_set/kf3d_age20_aff0.1_hit0_100m_803_pd.json
=> Begin evaluation...
Empty results
Empty results
Empty results
Empty results
Empty results
Empty results
Empty results
Empty results
Empty results
Empty results
Empty results

I'm not sure if there is something wrong from here, because the file generated looks fine:
kf3d_age20_aff0.1_hit0_100m_803_pd.json

It does show you the empty results for no ground truth annotation when evaluation. When you run the same code in the training split, it would show you the evaluation results.

{"timestamp": 174, "num": 174, "im_path": ["/media/huxi/DATA/inf_master/Semester-4/lecture/absolute/code/JM3DDT/3d-vehicle-tracking/3d-tracking/data/kitti_tracking/testing/image_02/0028/000174.png"], "class": "frame", "hypotheses": [{"height": 26.0, "width": 28.0, "trk_box": [622.1823653168534, 178.12736144976256, 671.7941276141136, 205.80495628396002], "det_box": [632.0, 178.0, 660.0, 204.0, 0.18501399457454681], "id": 1272, "x": 646, "y": 191, "dim": [1.4942878484725952, 1.6439176797866821, 3.850013494491577], "alpha": -1.2112747430801392, "roty": -1.1458846171921906, "depth": 41.02299230376193}, {"height": 60.0, "width": 63.0, "trk_box": [638.676846346, 175.13908976573774, 729.9815307848789, 230.30443618814388], "det_box": [647.0, 171.0, 710.0, 231.0, 0.999983012676239], "id": 1271, "x": 678, "y": 201, "dim": [1.532240390777588, 1.670041561126709, 4.048956394195557], "alpha": 1.440821886062622, "roty": 1.5270784997306373, "depth": 20.613992359586092}]}

1. PYTHONPATH=. python tools/convert_estimation_bdd.py kitti test --session 623 --epoch 100
   Here the result looks very weird
   0000_bdd_3d.json

{"id": -1, "category": "", "manualShape": false, "manualAttributes": false, "attributes": {"occluded": false, "truncated": false, "ignore": false}, "box2d": {"x1": 6, "y1": 223, "x2": 157, "y2": 369, "confidence": 0.177522}, "box3d": {"alpha": 0.0, "orientation": 0.0, "location": [0, 0, 0], "dimension": [0, 0, 0], "xc": -258, "yc": 360}}

the alpha is always 0.0 , the category is always empty

1. PYTHONPATH=. python tools/convert_tracking_bdd.py kitti test --session 623 --epoch 100
   the same: the alpha is always 0.0 , the category is always empty
   623_100_kitti_test_set/kf2ddeep_age20_aff0.1_hit0_100m_803/data/0000_bdd_3d.json

{"id": 28, "category": "", "manualShape": false, "manualAttributes": false, "attributes": {"occluded": false, "truncated": false, "ignore": false}, "box2d": {"x1": 972, "y1": 183, "x2": 1240, "y2": 375, "confidence": 0.999973}, "box3d": {"alpha": 0.0, "orientation": 0.0, "location": [[4.390286208707697, 1.1490172121191522, 3.698746681213379]], "dimension": [0, 0, 0], "xc": 1466, "yc": 397}}

So are there bugs in convert_estimation_bdd.py and convert_tracking_bdd.py, or I missed something?

Thank you!

The convert_estimation_bdd.py and convert_tracking_bdd.py is for validation only, not for the testing. The eval_dep_bdd.py and eval_mot_bdd.py can evaluate the output of what you have converted using depth and MOT metrics.

Instead, I would recommend you using visualize_kitti.py to both visualize and convert the output to KITTI acceptable format, so that you can submit them to the evaluation server.

from 3d-vehicle-tracking.