Code Monkey home page Code Monkey logo

Comments (3)

yanwei-li avatar yanwei-li commented on August 17, 2024

Hi, I never tested the result on nuscenes mini data. Could you reproduce the results on the nuscenes val set?

from uvtr.

zizo1111 avatar zizo1111 commented on August 17, 2024

Hi, I managed to free up enough space to download the whole dataset and I got similar results.

user@gpu-cluster:~/src/mmdet$ python3 extra_tools/test.py configs/uvtr/multi_modality/uvtr_m_v0075_r101_h5.py ckpts/uvtr/uvtr_m_v0075_r101_h5.pth --eval=bbox
/usr/local/lib/python3.8/dist-packages/mmcv/__init__.py:20: UserWarning: On January 1, 2023, MMCV will release v2.0.0, in which it will remove components related to the training process and add a data transformation module. In addition, it will rename the package names mmcv to mmcv-lite and mmcv-full to mmcv. See https://github.com/open-mmlab/mmcv/blob/master/docs/en/compatibility.md for more details.
  warnings.warn(
/usr/local/lib/python3.8/dist-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3190.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
load checkpoint from local path: ckpts/uvtr/uvtr_m_v0075_r101_h5.pth
2023-04-30 12:26:43,335 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.0.conv2 is upgraded to version 2.
2023-04-30 12:26:43,337 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.1.conv2 is upgraded to version 2.
2023-04-30 12:26:43,338 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.2.conv2 is upgraded to version 2.
2023-04-30 12:26:43,339 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.3.conv2 is upgraded to version 2.
2023-04-30 12:26:43,340 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.4.conv2 is upgraded to version 2.
2023-04-30 12:26:43,341 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.5.conv2 is upgraded to version 2.
2023-04-30 12:26:43,342 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.6.conv2 is upgraded to version 2.
2023-04-30 12:26:43,343 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.7.conv2 is upgraded to version 2.
2023-04-30 12:26:43,344 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.8.conv2 is upgraded to version 2.
2023-04-30 12:26:43,345 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.9.conv2 is upgraded to version 2.
2023-04-30 12:26:43,346 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.10.conv2 is upgraded to version 2.
2023-04-30 12:26:43,347 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.11.conv2 is upgraded to version 2.
2023-04-30 12:26:43,349 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.12.conv2 is upgraded to version 2.
2023-04-30 12:26:43,350 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.13.conv2 is upgraded to version 2.
2023-04-30 12:26:43,351 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.14.conv2 is upgraded to version 2.
2023-04-30 12:26:43,352 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.15.conv2 is upgraded to version 2.
2023-04-30 12:26:43,353 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.16.conv2 is upgraded to version 2.
2023-04-30 12:26:43,354 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.17.conv2 is upgraded to version 2.
2023-04-30 12:26:43,355 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.18.conv2 is upgraded to version 2.
2023-04-30 12:26:43,356 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.19.conv2 is upgraded to version 2.
2023-04-30 12:26:43,357 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.20.conv2 is upgraded to version 2.
2023-04-30 12:26:43,358 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.21.conv2 is upgraded to version 2.
2023-04-30 12:26:43,359 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.22.conv2 is upgraded to version 2.
2023-04-30 12:26:43,360 - root - INFO - ModulatedDeformConvPack img_backbone.layer4.0.conv2 is upgraded to version 2.
2023-04-30 12:26:43,362 - root - INFO - ModulatedDeformConvPack img_backbone.layer4.1.conv2 is upgraded to version 2.
2023-04-30 12:26:43,364 - root - INFO - ModulatedDeformConvPack img_backbone.layer4.2.conv2 is upgraded to version 2.
[                                                  ] 0/6019, elapsed: 0s, ETA:/usr/local/lib/python3.8/dist-packages/torch/nn/functional.py:4227: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
  warnings.warn(
[                                                  ] 1/6019, 0.3 task/s, elapsed: 3s, ETA: 18116s ~/src/mmdet/mmdet3d/core/bbox/coders/nms_free_coder.py:67: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  self.post_center_range = torch.tensor(
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 6019/6019, 2.1 task/s, elapsed: 2800s, ETA:     0s
Formating bboxes of pts_bbox
Start to convert detection format...
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 6019/6019, 40.5 task/s, elapsed: 149s, ETA:     0s
Results writes to /tmp/tmp2zhts1f2/results/pts_bbox/results_nusc.json
Evaluating bboxes of pts_bbox
mAP: 0.1348                                                                                               
mATE: 0.8543
mASE: 0.3209
mAOE: 0.6170
mAVE: 0.8633
mAAE: 0.2407
NDS: 0.2778
Eval time: 83.0s

Per-class results:
Object Class    AP      ATE     ASE     AOE     AVE     AAE
car     0.142   0.875   0.180   0.249   0.971   0.183
truck   0.049   0.992   0.265   0.324   0.876   0.229
bus     0.017   1.112   0.248   0.555   2.468   0.604
trailer 0.021   1.100   0.283   0.908   0.330   0.135
construction_vehicle    0.002   1.001   0.529   1.144   0.130   0.228
pedestrian      0.295   0.541   0.323   0.683   0.649   0.316
motorcycle      0.177   0.681   0.296   0.629   1.075   0.194
bicycle 0.253   0.663   0.311   0.703   0.408   0.038
traffic_cone    0.225   0.773   0.394   nan     nan     nan
barrier 0.166   0.803   0.380   0.359   nan     nan
{'pts_bbox_NuScenes/car_AP_dist_0.5': 0.0019, 'pts_bbox_NuScenes/car_AP_dist_1.0': 0.0591, 'pts_bbox_NuScenes/car_AP_dist_2.0': 0.191, 'pts_bbox_NuScenes/car_AP_dist_4.0': 0.316, 'pts_bbox_NuScenes/car_trans_err': 0.8745, 'pts_bbox_NuScenes/car_scale_err': 0.1798, 'pts_bbox_NuScenes/car_orient_err': 0.2486, 'pts_bbox_NuScenes/car_vel_err': 0.9711, 'pts_bbox_NuScenes/car_attr_err': 0.1828, 'pts_bbox_NuScenes/mATE': 0.8543, 'pts_bbox_NuScenes/mASE': 0.3209, 'pts_bbox_NuScenes/mAOE': 0.617, 'pts_bbox_NuScenes/mAVE': 0.8633, 'pts_bbox_NuScenes/mAAE': 0.2407, 'pts_bbox_NuScenes/truck_AP_dist_0.5': 0.0, 'pts_bbox_NuScenes/truck_AP_dist_1.0': 0.0057, 'pts_bbox_NuScenes/truck_AP_dist_2.0': 0.0585, 'pts_bbox_NuScenes/truck_AP_dist_4.0': 0.1328, 'pts_bbox_NuScenes/truck_trans_err': 0.9924, 'pts_bbox_NuScenes/truck_scale_err': 0.2653, 'pts_bbox_NuScenes/truck_orient_err': 0.3236, 'pts_bbox_NuScenes/truck_vel_err': 0.8758, 'pts_bbox_NuScenes/truck_attr_err': 0.2292, 'pts_bbox_NuScenes/construction_vehicle_AP_dist_0.5': 0.0, 'pts_bbox_NuScenes/construction_vehicle_AP_dist_1.0': 0.0, 'pts_bbox_NuScenes/construction_vehicle_AP_dist_2.0': 0.0002, 'pts_bbox_NuScenes/construction_vehicle_AP_dist_4.0': 0.0097, 'pts_bbox_NuScenes/construction_vehicle_trans_err': 1.0013, 'pts_bbox_NuScenes/construction_vehicle_scale_err': 0.5295, 'pts_bbox_NuScenes/construction_vehicle_orient_err': 1.144, 'pts_bbox_NuScenes/construction_vehicle_vel_err': 0.1305, 'pts_bbox_NuScenes/construction_vehicle_attr_err': 0.2276, 'pts_bbox_NuScenes/bus_AP_dist_0.5': 0.0, 'pts_bbox_NuScenes/bus_AP_dist_1.0': 0.0, 'pts_bbox_NuScenes/bus_AP_dist_2.0': 0.0076, 'pts_bbox_NuScenes/bus_AP_dist_4.0': 0.0606, 'pts_bbox_NuScenes/bus_trans_err': 1.1117, 'pts_bbox_NuScenes/bus_scale_err': 0.2478, 'pts_bbox_NuScenes/bus_orient_err': 0.5546, 'pts_bbox_NuScenes/bus_vel_err': 2.4679, 'pts_bbox_NuScenes/bus_attr_err': 0.6041, 'pts_bbox_NuScenes/trailer_AP_dist_0.5': 0.0, 'pts_bbox_NuScenes/trailer_AP_dist_1.0': 0.0, 'pts_bbox_NuScenes/trailer_AP_dist_2.0': 0.0116, 'pts_bbox_NuScenes/trailer_AP_dist_4.0': 0.072, 'pts_bbox_NuScenes/trailer_trans_err': 1.1001, 'pts_bbox_NuScenes/trailer_scale_err': 0.283, 'pts_bbox_NuScenes/trailer_orient_err': 0.908, 'pts_bbox_NuScenes/trailer_vel_err': 0.3297, 'pts_bbox_NuScenes/trailer_attr_err': 0.1348, 'pts_bbox_NuScenes/barrier_AP_dist_0.5': 0.011, 'pts_bbox_NuScenes/barrier_AP_dist_1.0': 0.0915, 'pts_bbox_NuScenes/barrier_AP_dist_2.0': 0.2187, 'pts_bbox_NuScenes/barrier_AP_dist_4.0': 0.3434, 'pts_bbox_NuScenes/barrier_trans_err': 0.8034, 'pts_bbox_NuScenes/barrier_scale_err': 0.3798, 'pts_bbox_NuScenes/barrier_orient_err': 0.3595, 'pts_bbox_NuScenes/barrier_vel_err': nan, 'pts_bbox_NuScenes/barrier_attr_err': nan, 'pts_bbox_NuScenes/motorcycle_AP_dist_0.5': 0.0253, 'pts_bbox_NuScenes/motorcycle_AP_dist_1.0': 0.133, 'pts_bbox_NuScenes/motorcycle_AP_dist_2.0': 0.2365, 'pts_bbox_NuScenes/motorcycle_AP_dist_4.0': 0.3129, 'pts_bbox_NuScenes/motorcycle_trans_err': 0.6813, 'pts_bbox_NuScenes/motorcycle_scale_err': 0.2961, 'pts_bbox_NuScenes/motorcycle_orient_err': 0.629, 'pts_bbox_NuScenes/motorcycle_vel_err': 1.0745, 'pts_bbox_NuScenes/motorcycle_attr_err': 0.1937, 'pts_bbox_NuScenes/bicycle_AP_dist_0.5': 0.0458, 'pts_bbox_NuScenes/bicycle_AP_dist_1.0': 0.1917, 'pts_bbox_NuScenes/bicycle_AP_dist_2.0': 0.3496, 'pts_bbox_NuScenes/bicycle_AP_dist_4.0': 0.4252, 'pts_bbox_NuScenes/bicycle_trans_err': 0.6635, 'pts_bbox_NuScenes/bicycle_scale_err': 0.311, 'pts_bbox_NuScenes/bicycle_orient_err': 0.7029, 'pts_bbox_NuScenes/bicycle_vel_err': 0.4081, 'pts_bbox_NuScenes/bicycle_attr_err': 0.0378, 'pts_bbox_NuScenes/pedestrian_AP_dist_0.5': 0.1057, 'pts_bbox_NuScenes/pedestrian_AP_dist_1.0': 0.264, 'pts_bbox_NuScenes/pedestrian_AP_dist_2.0': 0.3596, 'pts_bbox_NuScenes/pedestrian_AP_dist_4.0': 0.4512, 'pts_bbox_NuScenes/pedestrian_trans_err': 0.5412, 'pts_bbox_NuScenes/pedestrian_scale_err': 0.3233, 'pts_bbox_NuScenes/pedestrian_orient_err': 0.6827, 'pts_bbox_NuScenes/pedestrian_vel_err': 0.649, 'pts_bbox_NuScenes/pedestrian_attr_err': 0.3156, 'pts_bbox_NuScenes/traffic_cone_AP_dist_0.5': 0.0235, 'pts_bbox_NuScenes/traffic_cone_AP_dist_1.0': 0.1444, 'pts_bbox_NuScenes/traffic_cone_AP_dist_2.0': 0.2996, 'pts_bbox_NuScenes/traffic_cone_AP_dist_4.0': 0.4316, 'pts_bbox_NuScenes/traffic_cone_trans_err': 0.7734, 'pts_bbox_NuScenes/traffic_cone_scale_err': 0.3937, 'pts_bbox_NuScenes/traffic_cone_orient_err': nan, 'pts_bbox_NuScenes/traffic_cone_vel_err': nan, 'pts_bbox_NuScenes/traffic_cone_attr_err': nan, 'pts_bbox_NuScenes/NDS': 0.27776341220635314, 'pts_bbox_NuScenes/mAP': 0.13476739579111732}

Thanks in advance :)

from uvtr.

yanwei-li avatar yanwei-li commented on August 17, 2024

Hi, You can try v0.17.3 for the inference. I guess the mismatch is caused by the coordinate change in the new version. Of course, if you train it using v1.0.0rc5, it should achieve the correct results. Thanks!

from uvtr.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.